[R-meta] Dealing with missing variance in multilevel models: using number of replicates instead in ecological meta-analysis

[James Pustejovsky]

Gabriele,

I think using lme() is a very reasonable strategy in this case.

The main difference between models implemented in lme() versus those
implemented in metafor() is that lme() treats the residual variance at the
lowest level (so in this case, the variance of the effect size estimate,
nested within an experiment) as unknown and estimates based on a
homoskedasticity assumption or other some other posited structure, whereas
metafor treats the variances of the effect size estimates as known
quantities. In your situation, it seems that the variances really are
unknown, and so it would be appropriate to treat them as such, modeling
them as inversely proportional to the number of replicates. You might also
consider reporting cluster-robust variance estimates for the overall
average effect size estimate (or meta-regression coefficients, if you are
going to look at moderators too). This approach would allow for the
possibility that the assumption that the variances are inversely
proportional to the number of replicates is not correct.

Here's an example of what the model might look like--note too that I think
you need to use varFixed() rather than varIdent():

meta_fit <- lme(MSA~1, random = ~ 1 | Experiment, weights=varFixed(~ 1 /
Replicates), data=dat)

library(clubSandwich)
coef_test(meta_fit, vcov = "CR2") # this will automatically cluster on the
highest-level grouping variable,


Also, this approach could be extended to another level to model the
individual (species-specific) response ratios, nested within experiments,
rather than modeling only the aggregated mean species abundance estimates.
One potential advantage of doing so is that it would allow you to study
species-level moderators of effects.

James

On Fri, Oct 6, 2017 at 4:25 AM, Gabriele Midolo <gabriele.midolo at gmail.com>
wrote:

> Dear all,
>
> Do you think it is possible to perform a multilevel meta-analysis without
> sampling variance (from both the conceptual and the statistical point of
> view), using only the number of replicates to weight the importance of
> studies?
>
> I am using mean species abundance (MSA) as effect size, a quite unusual
> metric for biodiversity intactness in ecological meta-analysis (see an
> example in Benítez-López et al., 2010
> <http://serengetiwatch.org/wp-content/uploads/2010/08/
> Benitez_lopez-et-al_Roads_2010.pdf>).
> MSA is the mean of the response ratio of abundance values
> (Treatment/Control) for each individual species in a given ecological
> experiment/study. However, individual ratios are truncated at 1 if
> Treatment>Control, so MSA ranges between 0 and 1.
>
> Few studies included in my analysis report values of SD in the treatment
> and control plots. Additionaly, the type of data I am working with makes
> quite difficult -impossible, to my knowledge - to apply imputation
> techniques to estimate variance. However, all the studies included in the
> analysis report at least the number of replicates.
>
> Some of my co-authors suggested to weight the meta-analysis by the number
> of replicates, instead to limit the analysis to an "unweighted
> meta-analysis".
> Assuming this makes sense to you as well, how can I implement this in
> metafor?
>
> So far I am trying to arbitrarily assign the same variance to each study
> vi=1 and then use the W term to weight the importance by the number of
> replicates. Something like the following:
>
> dat$vi<-1
> res <- rma.mv(MSA, vi, W=Replicates, random =~1|Experiment/ID, data=dat)
>
> However, probably better, I am trying to fit something similar to a
> meta-analytic model with the following:
>
> lme(MSA~1, random =~1|Experiment/ID, weights=varIdent(~Replicates),
> data=dat)
>
> I am aware - based on Wolfgang's website - that fitting via lme() is not
> really the same than using 'rma.mv()' but maybe the latter is a more
> suitable model I am looking for given my specific case (?).
>
> (sorry - my first post here - let me know if I can share the data with you
> or if I missed something)
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis

	[[alternative HTML version deleted]]