[R-meta] Metafor and Robust() for hierarchical models

[James Pustejovsky]

Hi Cátia,

I'm not sure what you mean by checking the strength of the correlation
between effect sizes. If it's possible to get data about this correlation
or make an assumption that is supported by empirical evidence, then yes
definitely do so. But unfortunately, we are often in a situation where
there's not much information available about the correlations, and so we
need to make a somewhat arbitrary assumption and then support it with
sensitivity analysis.

James

On Fri, May 28, 2021 at 11:11 AM Cátia Ferreira De Oliveira <
cmfo500 using york.ac.uk> wrote:

> Thank you for your prompt reply!
>
> If you do not have an idea of the strength of the correlation between
> effect sizes within each study, would you suggest it being checked before
> using *impute_covariance_matrix() *function? Or would it be preferable to
> run the function for some of the possible correlations as is done in
> sensitivity analyses?
>
> Best wishes,
>
> Catia
>
> On Wed, 26 May 2021 at 16:58, James Pustejovsky <jepusto using gmail.com> wrote:
>
>> Hi Cátia,
>>
>> This is a good question. Robust variance estimation protects against
>> mis-specified assumptions regarding the sampling variances and covariances
>> of effect size estimates, as well as mis-specification of the random
>> effects structure of the model. Therefore, whether you to use it or not
>> depends on how much you trust the modeling assumptions that you're making.
>>
>> In the Konstantopoulos example, each sample provides one independent
>> effect size estimate, so there is not much reason to be concerned about
>> mis-specifiation of sampling variances or covariances of effect size
>> estimates. The random effects structure also seems pretty reasonable
>> (insofar as it captures the hierarchical structure of the data), so there
>> is not a particular reason to be concerned about mis-specification of that
>> aspect of the model. Therefore, RVE does not seem necessary here. You might
>> still prefer to use it if you are especially cautious or skeptical of
>> multi-level modeling assumptions in general, but I think there is a very
>> reasonable argument that it isn't necessary.
>>
>> James
>>
>> On Wed, May 26, 2021 at 10:19 AM Cátia Ferreira De Oliveira <
>> cmfo500 using york.ac.uk> wrote:
>>
>>> Hello,
>>>
>>> I have been going through some of the metafor resources and I was
>>> wondering
>>> if it would still be recommended to run the robust() from clubSandwich
>>> when
>>> having a dataset similar to this the one by Konstantopoulos or whether
>>> the
>>> approaches presented here
>>>
>>> https://www.rdocumentation.org/packages/metafor/versions/2.4-0/topics/dat.konstantopoulos2011
>>> are enough.
>>>
>>> Best wishes,
>>>
>>> Catia
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-sig-meta-analysis mailing list
>>> R-sig-meta-analysis using r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>
>>
>
> --
> Cátia Margarida Ferreira de Oliveira
> Psychology PhD Student
> Department of Psychology, Room B214
> University of York, YO10 5DD
>

	[[alternative HTML version deleted]]