[R-meta] Multiverse meta-analysis

Thu Feb 3 13:02:13 CET 2022

Hi,

there is a dedicated multiverse package, that might be of interest to 
you: https://cran.r-project.org/web/packages/multiverse/index.html
I haven't used it myself but there are several tutorials (vignettes).

Best,
-- 
Lukasz Stasielowicz
Osnabrück University
Institute for Psychology
Research methods, psychological assessment, and evaluation
Seminarstraße 20
49074 Osnabrück (Germany)

Am 03.02.2022 um 12:00 schrieb r-sig-meta-analysis-request using r-project.org:
> Send R-sig-meta-analysis mailing list submissions to
> 	r-sig-meta-analysis using r-project.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> or, via email, send a message with subject or body 'help' to
> 	r-sig-meta-analysis-request using r-project.org
> 
> You can reach the person managing the list at
> 	r-sig-meta-analysis-owner using r-project.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of R-sig-meta-analysis digest..."
> 
> 
> Today's Topics:
> 
>     1. Re: Meta-analysis of meta-analyses (James Pustejovsky)
>     2. Multiverse meta-analysis (Antonello Preti)
>     3. Re: [External] RE: Question about escalc, proportion ES, and
>        nested data (Harris, Jordan L)
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Wed, 2 Feb 2022 07:54:15 -0600
> From: James Pustejovsky <jepusto using gmail.com>
> To: =?utf-8?Q?"Dr._Gerta_R=C3=BCcker"?= <ruecker using imbi.uni-freiburg.de>
> Cc: "Viechtbauer, Wolfgang (SP)"
> 	<wolfgang.viechtbauer using maastrichtuniversity.nl>, Gladys Barragan-Jason
> 	<gladou86 using gmail.com>, R meta <r-sig-meta-analysis using r-project.org>
> Subject: Re: [R-meta] Meta-analysis of meta-analyses
> Message-ID: <9AC0DE96-CDEB-4367-8187-3877774DA999 using gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> I wonder about exactly this question for most of the second-order meta-analyses that I have seen. Beyond issues of statistical dependence, SOMA tend to further conceal heterogeneity of effects. Averaging together averages makes it that much harder to understand the extent to which effect sizes vary across studies.
> 
>> On Feb 2, 2022, at 4:37 AM, Dr. Gerta Rücker <ruecker using imbi.uni-freiburg.de> wrote:
> 
>> But you could easily identify the overlapping studies, couldn't you? Why not simply do a first-order meta-analysis of the union set of all studies found?
> 
> 
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Wed, 2 Feb 2022 18:02:09 +0100
> From: Antonello Preti <antoviral using gmail.com>
> To: r-sig-meta-analysis using r-project.org
> Subject: [R-meta] Multiverse meta-analysis
> Message-ID:
> 	<CAPmpGDsuKudPp6Su6yaXZZdXN1ogq-btr0hXwHA9+gnTG=1kRA using mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Hi everyone. We will see any soon some implementation of the so-called
> multiverse meta-analysis approach in 'meta' or 'metafor'?
> 
> Something like what was described in Voracek et al., 2019. Zeitschrift für
> Psychologie, 227(1), 64–82.
> https://doi.org/10.1027/2151-2604/a000357
> 
> The codes of Voracek are not easy to follow...
> 
> Antonello Preti
> 
> 	[[alternative HTML version deleted]]
> 
> 
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Wed, 2 Feb 2022 17:47:37 +0000
> From: "Harris, Jordan L" <jordan-l-harris using uiowa.edu>
> To: "Viechtbauer, Wolfgang (SP)"
> 	<wolfgang.viechtbauer using maastrichtuniversity.nl>
> Cc: "r-sig-meta-analysis using r-project.org"
> 	<r-sig-meta-analysis using r-project.org>
> Subject: Re: [R-meta] [External] RE: Question about escalc, proportion
> 	ES, and nested data
> Message-ID:
> 	<DM8PR04MB796095A511EEFA5B3EFA8668A6279 using DM8PR04MB7960.namprd04.prod.outlook.com>
> 	
> Content-Type: text/plain; charset="utf-8"
> 
> Hi Dr. Viechtbauer (and all),
> 
> Thank you very much for the reply!
> 
> I modeled the data using the available standardized loadings from the study's factor analysis. To derive a proportion of variance in each factor I squared the standardized loadings and averaged them. This was done for all of the specific factors (e.g., internalizing, and externalizing), and the general factor (sharing the same indicators as all the specific factors). I summed all these values (specific + general) to derive a "total" variance score, from which I divided the general variance score to calculate my variable of interest (i.e., general / general + specific). Unfortunately, I do not have standard errors from this method as I used excel functions to calculate these scores.
> 
> Given this context, do you have any recommendations for the measure and method by which I can derive effect size and sampling variance using escalc?
> 
> It might also help to know that I have sample size "sample_n" information from all unique timepoints and samples. The hope is that I can retain as much information as possible while also accounting for having the same participants assessed at multiple timepoints and not double counting them.
> 
> Any guidance will be appreciated!
> Best,
> Jordan
> ________________________________
> From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl>
> Sent: Wednesday, February 2, 2022 4:17 AM
> To: Harris, Jordan L <jordan-l-harris using uiowa.edu>; r-sig-meta-analysis using r-project.org <r-sig-meta-analysis using r-project.org>
> Subject: [External] RE: Question about escalc, proportion ES, and nested data
> 
> Dear Jordan,
> 
> Please see below for my responses.
> 
> Best,
> Wolfgang
> 
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>> Behalf Of Harris, Jordan L
>> Sent: Tuesday, 01 February, 2022 22:50
>> To: r-sig-meta-analysis using r-project.org
>> Subject: [R-meta] Question about escalc, proportion ES, and nested data
>>
>> Hi List Members,
>>
>> I am a graduate student who is new to R and meta-analyses, and I have been
>> running into problems getting my code sorted out.
>>
>> I am conducting a meta-analysis to explore how the structure of psychopathology
>> changes across childhood and adolescence. My effect size of interest is
>> represented by a proportion score that is conceptualized as ratio of variance
>> accounted for by a general factor, called "general_es" (i.e., general / general +
>> specific). These data do not currently have a sampling variance, nor have
>> transformed effect sizes been calculated. I have 3 levels of nested data: Level 1
>> = "timepoint_id", Level 2 = "sample_id", Level 1 = "study_id" which account for
>> non-independence of data. Here, I will call my data file "dat."
>>
>>   1.  How should I structure the escalc command to derive a "yi" and "V" values
>> needed for the rma.mv analysis? Would my measure be "PLO"?
> 
> "PLO" is for binomial data, which is not what you appear to have. A logit transformation may in itself be useful for proportions (however derived), but the calculation of the sampling variance in escalc() assumes that each proportion was calculated based on a random variable that follows a binomial distribution.
> 
> Ideally, one would need the standard errors of the proportions, which should come from whatever method/model was used to obtain those proportions. Then one can use the delta method to obtain the sampling variances of the logit-transformed proportions.
> 
> Getting the covariance between sampling errors would be even more difficult (multiple proportions obtained from the same sample will have non-zero correlations between the sampling errors).
> 
>>   2.  Would this structure be acceptable: rma.mv(yi, vi, random = ~ 1 |
>> study_id/sample_id/timepoint_id, data=dat)?
> 
> Possibly, but it is impossible to answer this properly without further details. For example, this model assumes constant correlation across timepoints, regardless of how far they are apart.
> 
> And as noted above, this model would not account for non-independent sampling errors.
> 
>> Thanks,
>> Jordan
> 
> 	[[alternative HTML version deleted]]
> 
> 
> 
> 
> ------------------------------
> 
> Subject: Digest Footer
> 
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
> 
> 
> ------------------------------
> 
> End of R-sig-meta-analysis Digest, Vol 57, Issue 6
> **************************************************