[R] Query regarding stats/p.adjust package (base) - specifically 'Hochberg' function

[David Swanepoel]

Dear R Core Dev Team, I hope all is well your side!
My apologies if this is not the correct point of contact to use to address this. If not, kindly advise or forward my request to the relevant team/persons.

I have a query regarding the 'Hochberg' method of the stats/p.adjust R package and hope you can assist me please. I have attached the data I used in Excel, which are lists of p-values for two different tests (Hardy Weinberg Equilibrium and Linkage Disequilibrium) for four population groups.

The basis of my concern is a discrepancy specifically between the Hochberg correction applied by four different R packages and the results of the Hochberg correction by the online tool, MultipleTesting.com<http://www.multipletesting.com/>.

Using the below R packages/functions, I ran multiple test correction (MTC) adjustments for the p-values listed in my dataset. All R packages below agreed with each other regarding the 'significance' of the p-values for the Hochberg adjustment.


  *   stats/p.adjust (method: Hochberg)
  *   mutoss/hochberg
  *   multtest/mt.rawp2adjp (procedure: Hochberg)
  *   elitism/mtp (method: Hochberg)

In checking the same values on the MultipleTesting.com, more p-values were flagged as significant for both the HWE and LD results across all four populations. I show these differences in the Excel sheet attached.
Essentially, using the R packages, only the first HWE p-value of Pop2 is significant at an alpha of 0.05. Using the MT.com tool, however, multiple p-values are shown to be significant across both tests with the Hochberg correction (the highlighted cells in the Excel sheet).


I asked the authors of MT.com about this, and they gave the following response:

"we have checked the issue, and we believe the computation by our page is correct (I cannot give opinion about the other packages).
When we look on the original Hochberg paper, and we only use the very first (smallest) p value, then m"=1, thus, according to the equation in the Hochberg 1988 paper, in this case practically there is no further correction necessary.
In other words, in case the *smallest* p value is smaller than alpha, then the *smallest* p value will remain significant irrespective of the other p values when we make the Hochberg correction."

I have attached the Hochberg paper here but, unfortunately, I don't understand enough of the stats to verify this. I have applied their logic on the same Excel sheet under the section "MT.com explanation", which shows why they consider the highlighted values significant.

I have also attached the 2 R files that I used to do the MTC runs and they can be run as is. They are just quite long as they contain many of the other MTC methods in the different packages too.

Kindly provide your thoughts as to whether you agree with this interpretation of the Hochberg paper or not? I would like to see concordance between the MT.com tool and the different R packages above (or understand why they are different), so that I can be more confident in the explanations of my own results as a stats layman.

I hope this makes sense. Please let me know if I need to clarify anything.


Many thanks and kind regards,
David

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Hochberg - 1988 - A Sharper Bonferroni Procedure for Multiple Tests of Significance.pdf
Type: application/pdf
Size: 325078 bytes
Desc: Hochberg - 1988 - A Sharper Bonferroni Procedure for Multiple Tests of Significance.pdf
URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20210824/19a37779/attachment.pdf>