[R-meta] Q-Statistic for Between-Subgroup Heterogeneity

Ty Beal tbe@l @ending from g@inhe@lth@org
Wed Jul 25 22:10:43 CEST 2018


Thanks, will post in plain text from now on. By simpler model, you mean the model with fixed tau^2? Just want to be sure.

On 7/25/18, 4:06 PM, "Viechtbauer, Wolfgang (SP)" <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

    Indeed, no sig. evidence for heteroscedasticity in the tau^2 values. Also, all of the information criteria (AIC, BIC, AICc) favor the simpler model.

    (please post in plain text -- then the formatting won't get messed up)

    Best,
    Wolfgang

    -----Original Message-----
    From: Ty Beal [mailto:tbeal using gainhealth.org]
    Sent: Wednesday, 25 July, 2018 21:57
    To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
    Subject: Re: [R-meta] Q-Statistic for Between-Subgroup Heterogeneity

    Sorry, the text did not format correctly. The pval is 0.3519.

    On 7/25/18, 3:54 PM, "Ty Beal" <tbeal using gainhealth.org> wrote:

    Thank you, Wolfgang. That clears things up.

    When I run the anova to compare models with fixed t^2 and variable t^2 I get the following:

            df       AIC      BIC      AICc  logLik    LRT   pval        QE
    Full    10 -101.7226 -79.6756  -97.7940 60.8613               8083.0222
    Reduced  6 -105.3006 -92.0724 -103.9006 58.6503 4.4219 0.3519 8083.0222

    I'm still not sure how to decide between models, but if I go by the P-value, there is no clear evidence that using the variable t^2 is appropriate, right?

    Best,
    Ty

    On 7/25/18, 3:19 PM, "Viechtbauer, Wolfgang (SP)" <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

    Hi Ty,

    Indeed, the test for between-group differences does not depend on which level of the factor is chosen as the reference level. For example:

    library(metafor)

    dat <- get(data(dat.bcg))
    dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)
    res <- rma(yi, vi, mods = ~ alloc, data=dat)
    res

    # Test of Moderators (coefficients 2:3):
    # QM(df = 2) = 1.7675, p-val = 0.4132

    res <- rma(yi, vi, mods = ~ relevel(factor(alloc), ref="random"), data=dat)
    res

    # Test of Moderators (coefficients 2:3):
    # QM(df = 2) = 1.7675, p-val = 0.4132

    As for your question what happens when you use '-1':

    res <- rma(yi, vi, mods = ~ alloc - 1, data=dat)
    res

    # Test of Moderators (coefficients 1:3):
    # QM(df = 3) = 15.9842, p-val = 0.0011

    This is testing whether the average true outcome is equal to 0 for all levels. That is something entirely different than testing if there are between-group differences.

    As for pooling vs. not pooling: The number 5 is arbitrary and not a hard threshold. With few studies, the estimate of tau^2 is imprecise. You can always do a model comparison: Fit the model with common tau^2 and the model with different tau^2 values per subgroup and then do a likelihood ratio test with anova():

    res1 <- rma.mv(yi, vi, mods = ~ alloc, random = ~ 1 | trial, data=dat)
    res1

    res2 <- rma.mv(yi, vi, mods = ~ alloc, random = ~ alloc | trial, struct="DIAG", data=dat)
    res2

    anova(res1, res2)

    In this case, there is no significant evidence for heteroscedasticity in the tau^2 values across subgroups (p = 0.8521).

    Best,
    Wolfgang

    -----Original Message-----
    From: Ty Beal [mailto:tbeal using gainhealth.org]
    Sent: Wednesday, 25 July, 2018 17:41
    To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
    Subject: Re: [R-meta] Q-Statistic for Between-Subgroup Heterogeneity

    Update.

    I read through Wolfgang's post here: http://www.metafor-project.org/doku.php/tips:multiple_factors_interactions

    I realized that when I changed the reference level to a different region, the QM statistic and P-value remains the same. The only thing that changes is the coefficients.

    I am still not sure what is being tested when I do the subgroup - 1, but I think it is not appropriate for my problem.

    Additionally, after reading Wolfgang's other post (http://www.metafor-project.org/doku.php/tips:comp_two_independent_estimates), I am now using rma.mv to allow a different tau^2 in each of the five regions.

    I was trying to decide whether or not to pool tau^2 and it's certainly ideal to not pool in this context, since I except the variance between regions to be different. I read general advice that calculating a separate tau^2 for each subgroup is not recommended if the subgroup contains 5 or less studies (https://www.meta-analysis.com/downloads/Meta-analysis%20Subgroups%20analysis.pdf). For two out of four questions (there will be 4 meta-analyses in total) one of the regions only has 5 studies. Does anyone know whether it would it be better to pool estimates of tau^2 or not?

    Best,
    Ty

    On 7/24/18, 9:07 PM, "Ty Beal" <tbeal using gainhealth.org> wrote:

    Dear Wolfgang,

    Thanks for the tips. I am still getting used to the mailing list. I read through the thread and still have a couple questions, which will require me to explain my analysis in more detail.

    I am conducting a meta-analysis of prevalence for four dietary questions based on frequency of consumption. For example, the first question is coded so that I am calculating the prevalence that individuals consume fruit less than once per day. My data include 72 countries with nationally representative data that are weighted according to a complex survey design. This requires me to use the survey package to calculate the prevalence and SE/CIs for each of the 72 countries.

    I have done the meta-analysis for all the studies together and calculated an I^2 statistic. Now I am making a forest plot of subgroups (5 regions--and later also by sex). I would like to present some measure of between subgroup heterogeneity, but I am not sure the best approach to take. I had been using the following code to obtain a Q-statistic:

    fruit_res_r <- rma(yi, vi, method="DL", mods = ~ factor(region), data=fruit)

    But after reading the post that Wolfgang added, I realized that I am comparing the first region as the reference level. When I use factor(region) - 1, it produces much higher Q-statistics and lower P-values. I just want to be sure this is the best approach for this context.

    Best,
    Ty

    On 7/24/18, 5:58 PM, "Viechtbauer, Wolfgang (SP)" <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

    Dear Ty,

    Please always CC the mailing list.

    And please re-read the posts under the link I posted (i.e., the entire thread). In particular, this one is most relevant:

    https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-December/000431.html

    It shows how to fit a model with different tau^2 values per subgroup using metafor. See also:

    http://www.metafor-project.org/doku.php/tips:comp_two_independent_estimates

    Best,
    Wolfgang

    -----Original Message-----
    From: Ty Beal [mailto:tbeal using gainhealth.org]
    Sent: Tuesday, 24 July, 2018 17:18
    To: Viechtbauer, Wolfgang (SP)
    Subject: RE: [R-meta] Q-Statistic for Between-Subgroup Heterogeneity

    Dear Wolfgang,

    If you do end up updating metafor to allow different between-subgroup heterogeneity, please let me know (if you remember). I may want to re-run the analysis to see how big of a difference it makes in uncertainty estimates.

    Best,
    Ty

    -----Original Message-----
    From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf Of Viechtbauer, Wolfgang (SP)
    Sent: Monday, July 16, 2018 8:22 AM
    To: r-sig-meta-analysis using r-project.org
    Subject: Re: [R-meta] Q-Statistic for Between-Subgroup Heterogeneity

    Good point. This was discussed before (for metagen(), but it's the same issue):

    https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-December/000430.html

    Best,
    Wolfgang

    -----Original Message-----
    From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Guido Schwarzer
    Sent: Monday, 16 July, 2018 13:13
    To: r-sig-meta-analysis using r-project.org
    Subject: Re: [R-meta] Q-Statistic for Between-Subgroup Heterogeneity

    Am 13.07.18 um 18:31 schrieb Viechtbauer, Wolfgang (SP):

    > [...]
    > In the output, you will find:
    >
    > Test of Moderators (coefficients 2:3):
    > QM(df = 2) = 1.7675, p-val = 0.4132
    >
    > This is the test for between-subgroup heterogeneity.

    ... assuming a common between-study variance tau2 across subgroups. The default in metaprop() of meta is to allow different between-study variances (analogous to RevMan 5, the software for Cochrane Reviews).
    However, this can be changed using argument 'tau.common = TRUE' in metaprop().

    As stated by Ty, metaprop() does not allow to specify the weights at the moment. I put this on my TODO list.

    Best wishes,
    Guido


[GAINbanner]<http://www.gainhealth.org>

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.

	[[alternative HTML version deleted]]



More information about the R-sig-meta-analysis mailing list