Are there other "controllers" similar to p-value, q-value, and power?

Metronome

Junior Member
Joined
Jun 12, 2018
Messages
128
According to the Wikipedia entry on q-values...

In statistical hypothesis testing, specifically multiple hypothesis testing, the q-value provides a means to control the positive false discovery rate (pFDR). Just as the p-value gives the expected false positive rate obtained by rejecting the null hypothesis for any result with an equal or smaller p-value, the q-value gives the expected pFDR obtained by rejecting the null hypothesis for any result with an equal or smaller q-value.

The "positive" qualifier here seems to be a distraction, as the article later states...

The q-value can be interpreted as the false discovery rate (FDR): the proportion of false positives among all positive results. Given a set of test statistics and their associated q-values, rejecting the null hypothesis for all tests whose q-value is less than or equal to some threshold [imath]\alpha[/imath] ensures that the expected value of the false discovery rate is [imath]\alpha[/imath].

So the q-value controls the False Discovery Rate. Similarly, the p-value appears to control the False Positive Rate, and the power appears to control the Sensitivity. This leaves five of the conditional probabilities unaccounted for: Positive Predictive Value, Specificity, Negative Predictive Value, False Omission Rate, and False Negative Rate.

Are there values which control these conditional probabilities as well, or alternatively, should we think of each controller as managing the tradeoff between two of the conditional probabilities, in which case, should we only expect a total of four controllers for the eight conditional probabilities?
 
According to the Wikipedia entry on q-values...



The "positive" qualifier here seems to be a distraction, as the article later states...



So the q-value controls the False Discovery Rate. Similarly, the p-value appears to control the False Positive Rate, and the power appears to control the Sensitivity. This leaves five of the conditional probabilities unaccounted for: Positive Predictive Value, Specificity, Negative Predictive Value, False Omission Rate, and False Negative Rate.

Are there values which control these conditional probabilities as well, or alternatively, should we think of each controller as managing the tradeoff between two of the conditional probabilities, in which case, should we only expect a total of four controllers for the eight conditional probabilities?
Have you seen Confusion Matrix (wiki)?

All of the other metrics are derived from [imath]\alpha, \beta[/imath].
Screen Shot 2023-01-31 at 6.52.53 AM.png
 
Have you seen Confusion Matrix (wiki)?

All of the other metrics are derived from [imath]\alpha, \beta[/imath].
View attachment 34910
Sure, I understand the matrix. I guess the confusing part for me is that p-value, q-value, and power in hypothesis testing seem to mean something more "epistemic" than the more "ontological" conditional probabilities False Positive Rate, False Discovery Rate, Sensitivity, Positive Predictive Value, Specificity, Negative Predictive Value, False Omission Rate, and False Negative Rate. There seems to be an "absence of evidence is not necessarily evidence of absence" chasm between the two sets of terminology; with a p-value, for example, one might not be able to reject a hypothesis but not be able to reject its negation either, whereas the appropriate pairs of "ontological" conditional probabilities are compliments and sum to [imath]1[/imath].
 
Top