Accuracy of a groups predictions

[Lord Vader]

Hi
I'm not math inclined so please keep your answers simple exponents, roots, factorial, Gaussian sum,... and logs on a good day is about as far as I go.
Ideally your answer would be in the form of a google sheets formula I could copy/paste.

A real world scenario....
Measure something, noting each measurement has some inaccuracy.
So, 2 measurements of something come back close to each other, the 3rd does not.
The average of the 2 good measurements is our answer

Importantly as the # of measurement devices increases, the likelihood that the average of the good is closer to actual increases.
aka I would trust 9 devices saying it's about this, rather than the 1 device out of 10 saying it's something else

..and my problem
A group of people, each person has 60% accuracy of predicting a value
How do I calculate the accuracy of the group as a whole.
..and red herring perhaps but it's a majority vote so odd number of people

Since accuracy must <= 100% I expect it to taper off as the number of people increases.
something like
1 = 60%
3 = 66%
....
9 = 99.99
21 = 99.999
etc etc

I searched but don't know the keywords for my search *angst*
I found 2 formulas, which I put in this sheet. But accuracy goes over 100% :/

Group Accuracy

docs.google.com

My thinking is:
Any formula must have the number of people and the accuracy as variables.
* If the number of people is 1 then the group accuracy can't be less than 60%
* The accuracy can not be greater than 100%
*..and if each persons accuracy is <50% then group accuracy would tend towards 0 is my thinking - I could be very very wrong, this is not my field of expertise


I'm modelling something, and I need something better than looking at a list of values and going "it's about ....there....I think"
each person is a model in the above problem

 

[stapel]

Hi
I'm not math inclined so please keep your answers simple exponents, roots, factorial, Gaussian sum,... and logs on a good day is about as far as I go.
Ideally your answer would be in the form of a google sheets formula I could copy/paste.

A real world scenario....
Measure something, noting each measurement has some inaccuracy.
So, 2 measurements of something come back close to each other, the 3rd does not.
The average of the 2 good measurements is our answer

Importantly as the # of measurement devices increases, the likelihood that the average of the good is closer to actual increases.
aka I would trust 9 devices saying it's about this, rather than the 1 device out of 10 saying it's something else

..and my problem
A group of people, each person has 60% accuracy of predicting a value
How do I calculate the accuracy of the group as a whole.
..and red herring perhaps but it's a majority vote so odd number of people

Since accuracy must <= 100% I expect it to taper off as the number of people increases.
something like
1 = 60%
3 = 66%
....
9 = 99.99
21 = 99.999
etc etc

I searched but don't know the keywords for my search *angst*
I found 2 formulas, which I put in this sheet. But accuracy goes over 100% :/

Group Accuracy

docs.google.com

My thinking is:
Any formula must have the number of people and the accuracy as variables.
* If the number of people is 1 then the group accuracy can't be less than 60%
* The accuracy can not be greater than 100%
*..and if each persons accuracy is <50% then group accuracy would tend towards 0 is my thinking - I could be very very wrong, this is not my field of expertise


I'm modelling something, and I need something better than looking at a list of values and going "it's about ....there....I think"
each person is a model in the above problem

Please provide the *exact* text of the project, a listing of recent topics of study (so we can see what tools you have and are probably expected to use), and a clear listing of what you have tried so far (even if you think it's totally wrong, so we can see what's going on).

Thank you!

 

[Dr.Peterson]

A group of people, each person has 60% accuracy of predicting a value

How do you define "60% accuracy"? Does it mean that 60% of the time they will be exactly right, or that they are always within 60% of the right value, or what?

it's a majority vote

How does "majority vote" work for predicting a value (in contrast to a yes/no question)? Are you referring to some unstated method to decide which "bad" values to ignore?

And where did you get those formulas? Their sources might help us see what you are trying to do (or not).

My sense is that you haven't defined your problem well enough yet.

 

[Lord Vader]

How do you define "60% accuracy"? Does it mean that 60% of the time they will be exactly right, or that they are always within 60% of the right value, or what?

How does "majority vote" work for predicting a value (in contrast to a yes/no question)? Are you referring to some unstated method to decide which "bad" values to ignore?

And where did you get those formulas? Their sources might help us see what you are trying to do (or not).

My sense is that you haven't defined your problem well enough yet.

Within 60% of the correct value, so 40% error.
Correct value is 1, they might answer .6 or 1.4

Bad values might be thought of as outliers.
The majority vote...if 6 people says "it's x", then you have 6 x values that are similar.
if 9 people says "it's x" then you have 9 x values that are similar
I'm looking to find a 'trust' value. If all of us are saying it's this then we can't all be wrong..and conversely I would be less inclined to trust our group answer if only 6 of us said it was x (or close to it)

My formulas came from ChatGPT because I don't know what I'm looking for - I don't know the jargon.

Oh yes, not well defined, which is why I came here to get experts to cut through the chaff

 

[stapel]

My formulas came from ChatGPT because I don't know what I'm looking for - I don't know the jargon.


Oh yes, not well defined, which is why I came here to get experts to cut through the chaff

"Not well-defined" means "doesn't have enough information to be able to answer, or isn't sufficiently clear as to allow for analysis, or perhaps is simply nonsensical". Unfortunately, it does not mean "somebody else can figure out all the missing information and create a solution". It doesn't work that way. Sorry.

 

[Dr.Peterson]

Within 60% of the correct value, so 40% error.
Correct value is 1, they might answer .6 or 1.4

Bad values might be thought of as outliers.
The majority vote...if 6 people says "it's x", then you have 6 x values that are similar.
if 9 people says "it's x" then you have 9 x values that are similar
I'm looking to find a 'trust' value. If all of us are saying it's this then we can't all be wrong..and conversely I would be less inclined to trust our group answer if only 6 of us said it was x (or close to it)

My formulas came from ChatGPT because I don't know what I'm looking for - I don't know the jargon.

Oh yes, not well defined, which is why I came here to get experts to cut through the chaff

I think you need to hire a statistician to look at your actual problem and work out an appropriate strategy. Don't imagine that ChatGPT can overcome your ignorance and figure out what you didn't know enough to say. Even a human may not be able to do that. (It is apparently doing exactly what you said you didn't want to do, saying "this sounds about right ... I think".)

Now, if you said that each prediction (measurement?) is normally distributed about the correct value, with a particular standard deviation (which might be derivable from your (very non-standard) "60%", with additional information), and if you defined outliers as values beyond a particular z score, and if you took the number of outliers into consideration in the calculation, there might be something that could be done. I don't know enough statistics at that level to be able to help.

There are several concepts that sound similar to what you are asking, such as the standard deviation of a sample mean, or the binomial distribution of the number of correct answers; but those require different data than you are providing, and don't quite fit your situation.