[R-meta] subgroup analysis using metabind

[Dr. Gerta Rücker]

Hi Eleanor,

Thank you for providing data and code. I run it and try to answer your questions.

1. Yes, the test for subgroup differences tests whether there are differences between the subgroups. Thus the null hypothesis is "there is no difference in effect between subgroups". Is this what you meant? I am not sure what your phrase "groupings within the subgroup" means.

For example, grouping by peat_type does not give a significant difference between subgroups bog and fen. The reason is that there is a lot of heterogeneity *within* each of the two subgroups, as you see with the forest plot:
> forest(m.cor.typ)
Heterogeneity within subgroups is as large as  that between groups.

It is different when grouping by depth: Now the test for subgroup differences gives p < 0.0001. Looking at the forest plot shows that correlations differ much between groups "21-30" and "11-20", with only small heterogeneity within those two groups; only within group "<10" we have still large heterogeneity. Try 
> forest(m.cor.dep)

2. The forest plots for the subgroup analysis provide I2 and tau2 values for the whole meta-analysis (I2 = 99%, tau2 = 4.9963) . These are just the same values as for the meta-analysis of all data, without any subgrouping. As said above, the covariate peat_type is not suitable to explain this heterogeneity, but depth is able. 

I am not fully sure what I2 and tau2 in the metabind summary show (I2 = 0, tau2 = 0.0683), but I assume that they describe (kind of) the heterogeneity between the confidence intervals (Cis) of the groups we see on the forest plot (which are largely overlapping). I am also not sure whether this makes much sense, as all the Cis come from the same data. I would just ignore these values and not show them, only the CIs. I would even omit metabind() here. Better to show the two forest plots separately.
 
3. I tried the same and got the same error message. Moreover, with the two subgroup analyses, p-values were given (despite the error message), but these p-values were completely wrong and didn't make sense! 

My advice at the moment is that you avoid adding p-values. In meta-analysis (and also statistics in general), p-values are more and more deprecated and they are not necessary. Best to stay with confidence intervals.

For the technical problem, we should wait for a response by Guido	.

Best,
Gerta



UNIVERSITÄTSKLINIKUM FREIBURG
Institute for Medical Biometry and Statistics

Dr. Gerta Rücker
Guest Scientist

Stefan-Meier-Straße 26 · 79104 Freiburg
gerta.ruecker using uniklinik-freiburg.de

https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker

-----Ursprüngliche Nachricht-----
Von: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> Im Auftrag von Eleanor Birch via R-sig-meta-analysis
Gesendet: Mittwoch, 25. September 2024 17:18
An: r-sig-meta-analysis using r-project.org
Cc: Eleanor Birch <Eleanor.Birch using ed.ac.uk>
Betreff: [R-meta] subgroup analysis using metabind

Hi

I've been using the meta package to do some subgroup analysis, and have come up with a few questions I hope someone can help me with. I am relatively new to using R, and this is my first foray into metaanalysis, so please excuse me if these have obvious answers, I have tried search on the mailing list with no obvious answer I could understand.

The code I have written performs two sub group analyses using metacor, then combines them using metabind and draws a forest plot of them. I have attached the simplified code, data, and metabind summary below, and attached the forest plot generated (with quite a bit of styling coding removed for simplicity).

My questions are:
1. I have included the test for effect in subgroup (test.effect.subgroup.random = TRUE). however, having read through the documentation about the package I am still not totally clear what this is testing. I think the null hypothesis is something like 'there is no different in effect between groupings within the subgroup'. could anyone confirm if this is roughly correct?

2. The I2 and Tau2 values in the metabind summary and forest plot are very different (99% and 0%, and 4.9963 and 0.0683, respectively). Is this correct? Am I doing something wrong? I don't have the same issue if I create a forest plot from a metacor object of the data.

3. I wanted to add the p-value for each row as a column. I tried adding it as a column on either side (leftcols = c('studlab','k', 'pval') or rightcols = c('pval', 'TE', 'ci')), both of which gives me the error "Error in x$labels[[i]] : subscript out of bounds".
I tried some problem solving: If i try added another item from the same metabind object (for example, 'statistic') in the same way, the forest plot is created with no problem. I called the data from pval outwith the forestplot (print(m.bind.sub$pval)) and it was printed no problem. I experimented with calling the data in a different way (leftcols = c('studlab','k', m.bind.sub$pval)) and got a different error "Error: Variable '0.550785867814187' not available in meta-analysis object.". I also tried rounding the p-values down (leftcols = c('studlab','k', (format(round(m.bind.sub$pval, 3), nsmall = 3)))) and got nearly the same error "Error: Variable '0.551' not available in meta-analysis object.".
Any advice to be able to add this column? It seems to be something of an issue with this data specifically, rather than the whole metabind object.

thank you for any and all advice
Eleanor Birch

Code and data:
#install library packagages
library(tidyverse)
library(meta)

#simplified data
res.all <- data.frame(paper = c("A", "A", "B", "C", "D", "D"), peat_type = c("bog","fen","fen","bog","bog","fen"), depth = c("<10", "21-30","21-30","<10", "11-20","11-20"), n = c(90,45,20,16,120,120), r = c(-0.556525043,0.748497322,0.526751218,0.99994506,-0.443646523,-0.520320445))

#meta analysis using a random effects model - peat type subgroup analysis
m.cor.typ <- metacor(data = res.all,
                 cor = r,
                 n = n,
                 studlab = paper,
                 fixed = FALSE,
                 random = TRUE,
                 method.tau = "REML",
                 method.random.ci = "HK",
                 keepdata = TRUE,
                 subgroup = peat_type,
                 tau.common = FALSE
                 )

#meta analysis using a random effects model - sample depth subgroup analysis
m.cor.dep <- metacor(data = res.all,
                 cor = r,
                 n = n,
                 studlab = paper,
                 fixed = FALSE,
                 random = TRUE,
                 method.tau = "REML",
                 method.random.ci = "HK",
                 keepdata = TRUE,
                 subgroup = depth,
                 tau.common = FALSE
                 )

# combining metaanalyses for different subgroups
m.bind.sub <- metabind(m.cor.typ, m.cor.dep,
                       name = c ("Peatland type", "Sample depth"),
                      random = TRUE)

summary(m.bind.sub)

# create forest plot for subgroup analysis
forest(m.bind.sub,
       random = TRUE, # use random effects model
       leftcols = c('studlab','k'),
       #rightcols = c('pval', 'TE', 'ci'),
       #leftcols = c('studlab','k', 'pval'),
       test.overall.random = TRUE, #include p-value test for overall summary
       test.effect.subgroup.random = TRUE, #include test for effect in subgroup using random model
       backtransf = TRUE
       )

metabind results summary:

          COR            95%-CI meta-analysis
bog    0.8797 [-1.0000; 1.0000] Peatland type
fen    0.3047 [-0.9362; 0.9814] Peatland type
<10    0.9803 [-1.0000; 1.0000]  Sample depth
21-30  0.6768 [-0.9131; 0.9966]  Sample depth
11-20 -0.4829 [-0.8218; 0.1082]  Sample depth

Number of studies: k = 6
                        COR            95%-CI    t p-value
Random effects model 0.6883 [-0.9079; 0.9967] 0.92  0.3997

Quantifying heterogeneity:
 tau^2 = 4.9963; tau = 2.2352; I^2 = 99.0% [98.6%; 99.3%]; H = 9.95 [8.49; 11.65]

Test of heterogeneity:
      Q d.f.  p-value
 494.75    5 < 0.0001

Results for meta-analyses (random effects model):
                k    COR            95%-CI  tau^2    tau      Q   I^2
Peatland type   6 0.6883 [-0.9079; 0.9967] 4.9963 2.2352 494.75 99.0%
Sample depth    6 0.6883 [-0.9079; 0.9967] 4.9963 2.2352 494.75 99.0%

Details on meta-analytical method:
- Inverse variance method
- Restricted maximum-likelihood estimator for tau^2
- Hartung-Knapp adjustment for random effects model (df = 5)
- Fisher's z transformation of correlations
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.