[R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11

Tarun Khanna kh@nn@ @end|ng |rom hert|e-@choo|@org
Tue Apr 21 16:44:20 CEST 2020 

Thank you, James. I understand that the moments estimators might be more useful when working with small sample sizes. In my data set I have about 60 studies and 110 effect sizes. So as such the dataset is not small. But I do want to estimate effect sizes for smaller sets of the data (there are multiple set of interventions which can be distinguished). In the smaller sets the number of effects decreases to as low as 15-30. In this context, I thought DL might be a better estimator. I will look into the robumeta package.


I also have a theoretical question  around RVE. The estimates that I get for RVE have much higher standard errors compared to the DL/REML estimator. I understand that this is to be expected, RVE is also likely to result in higher Type I errors. Is there any way to control for that in the metafor package?


Best

Tarun

Tarun Khanna
PhD Researcher

Hertie School

Friedrichstraße 180
10117 Berlin ∙ Germany
khanna using hertie-school.org ∙ www.hertie-school.org<http://www.hertie-school.org/>

________________________________
From: James Pustejovsky <jepusto using gmail.com>
Sent: 21 April 2020 16:22:38
To: Tarun Khanna
Cc: r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] R-sig-meta-analysis Digest, Vol 35, Issue 11

Hi Tarun,

Good question! The Dersimonian-Laird variance estimator is in a general class of what are called moment estimators. In principle, it is possible to use moment estimation for models that are more complex than the basic meta-analysis/meta-regression model (as estimated with rma.uni, for instance). In fact, this is precisely what the robumeta package does. The correlated effects model and hierarchical effects model implemented in that package use moment estimators of the between-study variance component, and for the hierarchical model, also the within-study variance component. As far as I understand them, I think these estimators are not as precise as using ML/REML, and are mainly intended as way to get "quick-and-dirty" values for use in a working model (which need not be super accurate, if RVE is then used to get standard errors/confidence intervals).

There has also been some statistical work on moment estimators for more complex multi-variate models:
* Chen, H., Manning, A. K., & Dupuis, J. (2012). A method of moments estimator for random effect multivariate meta‐analysis. Biometrics, 68(4), 1278-1284.
I'm not sure if the methods described here are implemented in software though.

Kind Regards,
James

On Tue, Apr 21, 2020 at 7:46 AM Tarun Khanna <khanna using hertie-school.org<mailto:khanna using hertie-school.org>> wrote:
Thanks for the excellent interpretation of RVE.


I was also wondering if it's possible to use DL method with RVE estimation in R? Obviously one can use this with the rma function but I cannot see similar options for the rma.mv<http://rma.mv> function, which only allows  "REML" or "ML" as the methods option.


Is there any theoretical reason why we cannot calculate DL with RVE? Or is it just that the functionality is not built into rma.mv<http://rma.mv> function in metafor?


Best

Tarun



Tarun Khanna

PhD Researcher

Hertie School


Friedrichstraße 180

10117 Berlin ∙ Germany
khanna using hertie-school.org<mailto:khanna using hertie-school.org> ∙ www.hertie-school.org<http://www.hertie-school.org><http://www.hertie-school.org/>

________________________________
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org<mailto:r-sig-meta-analysis-bounces using r-project.org>> on behalf of r-sig-meta-analysis-request using r-project.org<mailto:r-sig-meta-analysis-request using r-project.org> <r-sig-meta-analysis-request using r-project.org<mailto:r-sig-meta-analysis-request using r-project.org>>
Sent: 18 April 2020 12:00:02
To: r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>
Subject: R-sig-meta-analysis Digest, Vol 35, Issue 11

Send R-sig-meta-analysis mailing list submissions to
        r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>

To subscribe or unsubscribe via the World Wide Web, visit
        https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
or, via email, send a message with subject or body 'help' to
        r-sig-meta-analysis-request using r-project.org<mailto:r-sig-meta-analysis-request using r-project.org>

You can reach the person managing the list at
        r-sig-meta-analysis-owner using r-project.org<mailto:r-sig-meta-analysis-owner using r-project.org>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of R-sig-meta-analysis digest..."


Today's Topics:

   1. Re: Robust variance estimation (James Pustejovsky)

----------------------------------------------------------------------

Message: 1
Date: Fri, 17 Apr 2020 12:47:11 -0500
From: James Pustejovsky <jepusto using gmail.com<mailto:jepusto using gmail.com>>
To: Emily Russell <emilyrussell99 using outlook.com<mailto:emilyrussell99 using outlook.com>>
Cc: "r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>"
        <r-sig-meta-analysis using r-project.org<mailto:r-sig-meta-analysis using r-project.org>>
Subject: Re: [R-meta] Robust variance estimation
Message-ID:
        <CAFUVuJwOX6uORO7BUxdyEtq+Mw8P3UiSRe9mDmYYdJ0u8+e=rg using mail.gmail.com<mailto:rg using mail.gmail.com>>
Content-Type: text/plain; charset="utf-8"

Hi Emily,

I think one useful intuition about robust variance estimation is that its a
way of capturing the uncertainty in an estimate *using only between-study
variation*.

The RVE approach is analogous to how one would calculate the standard error
of a mean from a simple random sample, so it's helpful to review that
first. Say that we have a random sample of N observations, Y1, Y2,...., YN,
and we're trying to estimate the population mean from this sample. The
usual estimator is the sample mean, Y-bar. The standard error of Y-bar is
SE = SD / sqrt(N), where SD is the standard deviation of the sample
observations.

Okay so now let's think about the meta-analysis context. Say that you have
a meta-analysis with multiple effect sizes, which could be correlated,
nested within a set of studies, which can be treated as independent of each
other. And say that our goal is just to estimate the population mean effect
size across the set of studies.

The alternative to RVE is to use a "model-based" approach to uncertainty
estimation (like a multi-variate or hierarchical model). To do that
properly, we have to come up with an appropriate model for how the effect
sizes are related to each other (i.e., how they correlate) within each
study, and then also how they vary across studies. In other words, we have
to have a model for both the within-study variation (and covariation) and
the between-study variation. We use this model for the within- and
between-study variation to determine how to take a weighted average of all
of the effect sizes. And then, in the model-based approach, we also use it
to determine a standard error for the weighted average. As a result, the
accuracy of the standard error *is contingent on the modeling assumptions
being appropriate*.

The RVE approach still uses a model to determine how to take a weighted
average of all of the effect sizes, but it does not rely on the model for
assessing the uncertainty of the average. Instead, it just uses the between
study variation. Behind the RVE formulas are really two steps of
calculation. First is to calculate an average effect size for each study.
Since studies are independent, each of these average effects can be treated
as independent. And the overall average is just an average of the
study-specific average effect sizes (albeit with weights involved). So
actually, we're in a situation that's very similar to taking the mean of a
simple random sample, only now our "observations" are study-specific
average effect size estimates. Consequently, the second step in the RVE
standard error calculation is to take the SD of the study-specific average
effect sizes, then dividing the square root of the number of studies.
(Again, in practice there's weights involved, but the intuition is still
the same.)

There are two key advantage of this approach. One is that it works fine for
most any set of weights we might use in calculating the overall average
effect size. The weights don't have to be exactly right or optimal in any
sense. The second is that we can do these calculations without knowing
exactly how the individual effect sizes within each study are correlated
with each other. All we need is to be able to calculate study-specific
average effect sizes. So we don't have to rely on our modeling assumptions
being exactly right/accurate in order to trust the standard errors from
RVE.

The intuition about using only between-study variation can actually be
carried further to more complex scenarios with meta-regression on a set of
covariates, too.

Kind Regards,
James

On Fri, Apr 17, 2020 at 5:54 AM Emily Russell <emilyrussell99 using outlook.com<mailto:emilyrussell99 using outlook.com>>
wrote:

> Dear Friends and Colleagues
>
> I hope this is not too basic a question; but could someone give me an
> intuitive rather than technical explanation of what robust variance
> estimation does (as in robu in robumeta and robust in metafor)?  I have
> looked at the papers referred to but they are a bit 'heavy' for me.
>
> Thank you so much
>
> Emily
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org<mailto:R-sig-meta-analysis using r-project.org>
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>

        [[alternative HTML version deleted]]




------------------------------

Subject: Digest Footer

_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis using r-project.org<mailto:R-sig-meta-analysis using r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis


------------------------------

End of R-sig-meta-analysis Digest, Vol 35, Issue 11
***************************************************

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis using r-project.org<mailto:R-sig-meta-analysis using r-project.org>
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis

	[[alternative HTML version deleted]]

More information about the R-sig-meta-analysis mailing list