[R-meta] Q-Statistic for Between-Subgroup Heterogeneity

[Ty Beal]

Sorry, the text did not format correctly. The pval is 0.3519.

On 7/25/18, 3:54 PM, "Ty Beal" <tbeal using gainhealth.org> wrote:

    Thank you, Wolfgang. That clears things up.

    When I run the anova to compare models with fixed t^2 and variable t^2 I get the following:

                        df               AIC               BIC            AICc           logLik           LRT        pval                  QE
    Full             10    -101.7226    -79.6756     -97.7940      60.8613                                     8083.0222
    Reduced      6    -105.3006    -92.0724     -103.9006    58.6503    4.4219    0.3519    8083.0222

    I'm still not sure how to decide between models, but if I go by the P-value, there is no clear evidence that using the variable t^2 is appropriate, right?

    Best,
    Ty

    On 7/25/18, 3:19 PM, "Viechtbauer, Wolfgang (SP)" <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

        Hi Ty,

        Indeed, the test for between-group differences does not depend on which level of the factor is chosen as the reference level. For example:

        library(metafor)

        dat <- get(data(dat.bcg))
        dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg)
        res <- rma(yi, vi, mods = ~ alloc, data=dat)
        res

        # Test of Moderators (coefficients 2:3):
        # QM(df = 2) = 1.7675, p-val = 0.4132

        res <- rma(yi, vi, mods = ~ relevel(factor(alloc), ref="random"), data=dat)
        res

        # Test of Moderators (coefficients 2:3):
        # QM(df = 2) = 1.7675, p-val = 0.4132

        As for your question what happens when you use '-1':

        res <- rma(yi, vi, mods = ~ alloc - 1, data=dat)
        res

        # Test of Moderators (coefficients 1:3):
        # QM(df = 3) = 15.9842, p-val = 0.0011

        This is testing whether the average true outcome is equal to 0 for all levels. That is something entirely different than testing if there are between-group differences.

        As for pooling vs. not pooling: The number 5 is arbitrary and not a hard threshold. With few studies, the estimate of tau^2 is imprecise. You can always do a model comparison: Fit the model with common tau^2 and the model with different tau^2 values per subgroup and then do a likelihood ratio test with anova():

        res1 <- rma.mv(yi, vi, mods = ~ alloc, random = ~ 1 | trial, data=dat)
        res1

        res2 <- rma.mv(yi, vi, mods = ~ alloc, random = ~ alloc | trial, struct="DIAG", data=dat)
        res2

        anova(res1, res2)

        In this case, there is no significant evidence for heteroscedasticity in the tau^2 values across subgroups (p = 0.8521).

        Best,
        Wolfgang

        -----Original Message-----
        From: Ty Beal [mailto:tbeal using gainhealth.org]
        Sent: Wednesday, 25 July, 2018 17:41
        To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis using r-project.org
        Subject: Re: [R-meta] Q-Statistic for Between-Subgroup Heterogeneity

        Update.

        I read through Wolfgang's post here: http://www.metafor-project.org/doku.php/tips:multiple_factors_interactions

        I realized that when I changed the reference level to a different region, the QM statistic and P-value remains the same. The only thing that changes is the coefficients.

        I am still not sure what is being tested when I do the subgroup - 1, but I think it is not appropriate for my problem.

        Additionally, after reading Wolfgang's other post (http://www.metafor-project.org/doku.php/tips:comp_two_independent_estimates), I am now using rma.mv to allow a different tau^2 in each of the five regions.

        I was trying to decide whether or not to pool tau^2 and it's certainly ideal to not pool in this context, since I except the variance between regions to be different. I read general advice that calculating a separate tau^2 for each subgroup is not recommended if the subgroup contains 5 or less studies (https://www.meta-analysis.com/downloads/Meta-analysis%20Subgroups%20analysis.pdf). For two out of four questions (there will be 4 meta-analyses in total) one of the regions only has 5 studies. Does anyone know whether it would it be better to pool estimates of tau^2 or not?

        Best,
        Ty

        On 7/24/18, 9:07 PM, "Ty Beal" <tbeal using gainhealth.org> wrote:

        Dear Wolfgang,

        Thanks for the tips. I am still getting used to the mailing list. I read through the thread and still have a couple questions, which will require me to explain my analysis in more detail.

        I am conducting a meta-analysis of prevalence for four dietary questions based on frequency of consumption. For example, the first question is coded so that I am calculating the prevalence that individuals consume fruit less than once per day. My data include 72 countries with nationally representative data that are weighted according to a complex survey design. This requires me to use the survey package to calculate the prevalence and SE/CIs for each of the 72 countries.

        I have done the meta-analysis for all the studies together and calculated an I^2 statistic. Now I am making a forest plot of subgroups (5 regions--and later also by sex). I would like to present some measure of between subgroup heterogeneity, but I am not sure the best approach to take. I had been using the following code to obtain a Q-statistic:

        fruit_res_r <- rma(yi, vi, method="DL", mods = ~ factor(region), data=fruit)

        But after reading the post that Wolfgang added, I realized that I am comparing the first region as the reference level. When I use factor(region) - 1, it produces much higher Q-statistics and lower P-values. I just want to be sure this is the best approach for this context.

        Best,
        Ty

        On 7/24/18, 5:58 PM, "Viechtbauer, Wolfgang (SP)" <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:

        Dear Ty,

        Please always CC the mailing list.

        And please re-read the posts under the link I posted (i.e., the entire thread). In particular, this one is most relevant:

        https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-December/000431.html

        It shows how to fit a model with different tau^2 values per subgroup using metafor. See also:

        http://www.metafor-project.org/doku.php/tips:comp_two_independent_estimates

        Best,
        Wolfgang

        -----Original Message-----
        From: Ty Beal [mailto:tbeal using gainhealth.org]
        Sent: Tuesday, 24 July, 2018 17:18
        To: Viechtbauer, Wolfgang (SP)
        Subject: RE: [R-meta] Q-Statistic for Between-Subgroup Heterogeneity

        Dear Wolfgang,

        If you do end up updating metafor to allow different between-subgroup heterogeneity, please let me know (if you remember). I may want to re-run the analysis to see how big of a difference it makes in uncertainty estimates.

        Best,
        Ty

        -----Original Message-----
        From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf Of Viechtbauer, Wolfgang (SP)
        Sent: Monday, July 16, 2018 8:22 AM
        To: r-sig-meta-analysis using r-project.org
        Subject: Re: [R-meta] Q-Statistic for Between-Subgroup Heterogeneity

        Good point. This was discussed before (for metagen(), but it's the same issue):

        https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2017-December/000430.html

        Best,
        Wolfgang

        -----Original Message-----
        From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On Behalf Of Guido Schwarzer
        Sent: Monday, 16 July, 2018 13:13
        To: r-sig-meta-analysis using r-project.org
        Subject: Re: [R-meta] Q-Statistic for Between-Subgroup Heterogeneity

        Am 13.07.18 um 18:31 schrieb Viechtbauer, Wolfgang (SP):

        > [...]
        > In the output, you will find:
        >
        > Test of Moderators (coefficients 2:3):
        > QM(df = 2) = 1.7675, p-val = 0.4132
        >
        > This is the test for between-subgroup heterogeneity.

        ... assuming a common between-study variance tau2 across subgroups. The default in metaprop() of meta is to allow different between-study variances (analogous to RevMan 5, the software for Cochrane Reviews).
        However, this can be changed using argument 'tau.common = TRUE' in metaprop().

        As stated by Ty, metaprop() does not allow to specify the weights at the moment. I put this on my TODO list.

        Best wishes,
        Guido




[GAINbanner]<http://www.gainhealth.org>

Disclaimer

The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.

	[[alternative HTML version deleted]]