[R-meta] SEM of correlational meta-analytic data?

[Lukasz Stasielowicz]

Dear Gladys,

you write "most of the correlational studies I included in the 
meta-analysis (from which I extracted Pearson correlations) also 
performed a SEM showing that Human-nature connectedness mediates the 
effect."
Please correct me if I'm wrong but I'm getting the impression that 
you're assuming that building mediation models within the SEM framework 
automatically enables one making causal claims.
As Wolfgang has already said: causality is tricky.

In short, there are three main assumptions in social sciences:

1. Covariance. Checked - you mention that the relevant variables are 
correlated.

2. Excluding alternative explanations. Although it is possible to 
increase the plausibility that this assumption is met even when using 
correlational data (e.g., adjusting for confounding variables, 
propensity score matching) but it would require meta-analyzing 
regression weights, which is rather tricky in the meta-analytic context, 
considering that different researchers adjust for different variables. 
Thus, it is difficult to argue that this assumption is met when 
conducting MASEM using data from observational studies.

3. Time precedence: The Cause precedes the effect. In case of mediation 
models it usually requires using a longitudinal design with at least 
three time points and assessing each variable at each measurement 
occassion. However, there are some alternatives (e.g., half-longitudinal 
designs, using state variables rather than trait variables). 
Unfortunately, many (most?) mediation models violate this assumption.

This is well-documented See e.g.,
Fiedler , K., Harris, C., & Schott, M. (2018). Unwarranted inferences 
from statistical mediation tests. An analysis of articles published in 
2015. Journal of Experimental Social Psychology , 75 , 95 102. http:// 
doi.org/10.1016/j.jesp.2017.11.008
Lemmer , G., & Gollwitzer, M. (2017). The true ” indirect effect won’t 
(always) stand up : When and why reverse mediation testing fails . 
Journal of Experimental Social Psychology , 69 , 144 149. http:// 
doi.org/10.1016/j.jesp.2016.05.002
Pek , J., & Hoyle , R. H. (2016). On the ( validity of tests of simple 
mediation : Threats and solutions . Social and Personality Psychology 
Compass , 10 (3), 150 163. http://doi.org/10.1111/spc3.12237
Pirlott , A. G., & MacKinnon, D. P. (2016). Design approaches to 
experimental mediation . Journal of Experimental Social Psychology , 66 
, 29 38. http://doi.org/10.1016/j.jesp.2015.09.012

One could potentially use strict exclusion criteria in the meta-analysis 
to increase the plausibility of causality claims (e.g., only include 
longitudinal primary studies), but it is probably not feasible as there 
are probably not many studies that meet such strict criteria.

Summing up, due to the violation of assumptions #2 and #3 it is seldom 
possible to make causality claims when conducting a meta-analysis.



best wishes,
Lukasz
-- 
Lukasz Stasielowicz
Osnabrück University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Seminarstraße 20
49074 Osnabrück (Germany)




Am 18.01.2021 um 09:00 schrieb r-sig-meta-analysis-request using r-project.org:
> Send R-sig-meta-analysis mailing list submissions to
> 	r-sig-meta-analysis using r-project.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> or, via email, send a message with subject or body 'help' to
> 	r-sig-meta-analysis-request using r-project.org
>
> You can reach the person managing the list at
> 	r-sig-meta-analysis-owner using r-project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-meta-analysis digest..."
>
>
> Today's Topics:
>
>     1. Re: SEM of correlational meta-analytic data?
>        (Viechtbauer, Wolfgang (SP))
>     2. Re: SEM of correlational meta-analytic data?
>        (Gladys Barragan-Jason)
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 17 Jan 2021 11:11:14 +0000
> From: "Viechtbauer, Wolfgang (SP)"
> 	<wolfgang.viechtbauer using maastrichtuniversity.nl>
> To: Gladys Barragan-Jason <gladou86 using gmail.com>,
> 	"r-sig-meta-analysis using r-project.org"
> 	<r-sig-meta-analysis using r-project.org>
> Subject: Re: [R-meta] SEM of correlational meta-analytic data?
> Message-ID: <758f447c2e04453ebc684b0e187e47fc using UM-MAIL3214.unimaas.nl>
> Content-Type: text/plain; charset="us-ascii"
>
> Dear Gladys,
>
> Inferring causality from observational data is tricky business. SEM (with primary data) or meta-analytic structural equation modeling (MASEM) does not magically allow us to do so just by fitting some model.
>
> But if you want to do MASEM, then the MetaSEM package is a good choice. I also recently added some functionality to metafor that goes a bit in the same direction. See:
>
> https://wviechtb.github.io/metafor/reference/rcalc.html
> https://wviechtb.github.io/metafor/reference/matreg.html
>
> Note that you will need to install the 'devel' version of metafor to make use of these functions:
>
> https://wviechtb.github.io/metafor/index.html#installation
>
> Best,
> Wolfgang
>
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org]
>> On Behalf Of Gladys Barragan-Jason
>> Sent: Sunday, 17 January, 2021 11:23
>> To: r-sig-meta-analysis using r-project.org
>> Subject: [R-meta] SEM of correlational meta-analytic data?
>>
>> Dear all,
>>
>> I am conducting a meta-analysis on the causes and consequences of
>> human-nature connectedness. As most of the studies were correlational, I
>> collected zero order Pearson r correlations between HNC and let's say 3
>> moderators (Exposure to nature, human-welfare and nature conservation). I
>> was able to obtain positive and moderate estimates in running one model by
>> moderator with lab and study as random effect thanks to the rma.mv
>> function which was great.
>>
>> My only concern now if whether we could somehow infer causality from those
>> meta-analytic data in making Structural Equation Modelling (SEM) on those
>> data. I saw that the MetaSEM package can do so but I have the feeling that
>> it is not using the same structure/function as metafor (e.g. meta3 instead
>> of rma.mv) leading to some discrepancies.
>>
>> I would like to know if someone has developed a package or a function to do
>> this type of causal analysis from meta-analytic correlation data.
>>
>> The aim would be validate (or invalidate) a model where exposure to nature
>> increases HNC which in turn increases Nature conservation and welfare
>> (rather than the opposite). I don(t know if it is feasible but would be
>> great if so.
>>
>> Any advice would be more than welcome :-)
>>
>> All the best,
>>
>> Gladys
>
>
>
> ------------------------------
>
> Message: 2
> Date: Mon, 18 Jan 2021 09:00:10 +0100
> From: Gladys Barragan-Jason <gladou86 using gmail.com>
> To: "Viechtbauer, Wolfgang (SP)"
> 	<wolfgang.viechtbauer using maastrichtuniversity.nl>,
> 	r-sig-meta-analysis using r-project.org
> Subject: Re: [R-meta] SEM of correlational meta-analytic data?
> Message-ID:
> 	<CAGtj4Dxy_oYB63E6BV80be6Jd9e8WAXHe=p_Kn9wfvb1xbjkyQ using mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Dear Wolfgang,
> Thanks for your helpful reply. Actually I am not (randomly) assuming the
> causality. For instance, most of the correlational studies I included in
> the meta-analysis (from which I extracted Pearson correlations) also
> performed a SEM showing that Human-nature connectedness mediates the
> effect. Would reporting how many papers actually report such causation
> and/or making a meta-analysis on the extracted beta would make more sense?
> For the latter possibility, another problem is that the number of
> moderators included in the the SEM would differ between studies...
> What do you think?
> Thanks a lot for your reply.
> Best,
> Gladys
>
> Le dim. 17 janv. 2021 à 12:11, Viechtbauer, Wolfgang (SP) <
> wolfgang.viechtbauer using maastrichtuniversity.nl> a écrit :
>
>> Dear Gladys,
>>
>> Inferring causality from observational data is tricky business. SEM (with
>> primary data) or meta-analytic structural equation modeling (MASEM) does
>> not magically allow us to do so just by fitting some model.
>>
>> But if you want to do MASEM, then the MetaSEM package is a good choice. I
>> also recently added some functionality to metafor that goes a bit in the
>> same direction. See:
>>
>> https://wviechtb.github.io/metafor/reference/rcalc.html
>> https://wviechtb.github.io/metafor/reference/matreg.html
>>
>> Note that you will need to install the 'devel' version of metafor to make
>> use of these functions:
>>
>> https://wviechtb.github.io/metafor/index.html#installation
>>
>> Best,
>> Wolfgang
>>
>>> -----Original Message-----
>>> From: R-sig-meta-analysis [mailto:
>> r-sig-meta-analysis-bounces using r-project.org]
>>> On Behalf Of Gladys Barragan-Jason
>>> Sent: Sunday, 17 January, 2021 11:23
>>> To: r-sig-meta-analysis using r-project.org
>>> Subject: [R-meta] SEM of correlational meta-analytic data?
>>>
>>> Dear all,
>>>
>>> I am conducting a meta-analysis on the causes and consequences of
>>> human-nature connectedness. As most of the studies were correlational, I
>>> collected zero order Pearson r correlations between HNC and let's say 3
>>> moderators (Exposure to nature, human-welfare and nature conservation). I
>>> was able to obtain positive and moderate estimates in running one model by
>>> moderator with lab and study as random effect thanks to the rma.mv
>>> function which was great.
>>>
>>> My only concern now if whether we could somehow infer causality from those
>>> meta-analytic data in making Structural Equation Modelling (SEM) on those
>>> data. I saw that the MetaSEM package can do so but I have the feeling that
>>> it is not using the same structure/function as metafor (e.g. meta3 instead
>>> of rma.mv) leading to some discrepancies.
>>>
>>> I would like to know if someone has developed a package or a function to
>> do
>>> this type of causal analysis from meta-analytic correlation data.
>>>
>>> The aim would be validate (or invalidate) a model where exposure to nature
>>> increases HNC which in turn increases Nature conservation and welfare
>>> (rather than the opposite). I don(t know if it is feasible but would be
>>> great if so.
>>>
>>> Any advice would be more than welcome :-)
>>>
>>> All the best,
>>>
>>> Gladys
>