[R-meta] Studies with independent samples of participants

Dr. Gerta Rücker ruecker @end|ng |rom |mb|@un|-|re|burg@de
Mon Jun 21 23:17:31 CEST 2021


Dear Jack,

Is this distinction between different definitions of "control" also made 
in other studies? That is, are there throughout two (or more) types of 
control?

If so, this looks a bit like a network meta-analysis could also be 
appropriate.

Best,

Gerta

Am 21.06.2021 um 22:48 schrieb Jack Solomon:
> Much appreciated, James. There is one more complication in these two
> studies. They have used two comparison/control groups (as the definition of
> 'control' has been debated in the literature I'm meta-analyzing).
>
> Given that, in addition to the 'sample' column, should I create an
> additional column ('control') to distinguish between effect sizes (SMDs in
> this case) that have been obtained by comparing the treated groups to
> control 1 vs. control 2 (see below)?
>
> If yes, then, does the addition of a 'control' column call for the addition
> of a random effect for 'control' as well (again, something to be
> empirically tested)?
>
> Thanks again, Jack
>
> ***consider 'sample' & 'control' in coding:
> study sample es  control
> 1         1         .1      1
> 1         1         .2      2
> 1         2         .3      1
> 1         2         .4      2
> 2         1         .5      1
> 2         2         .6      2
> 3         1         .7      1
>
> On Mon, Jun 21, 2021 at 3:25 PM James Pustejovsky <jepusto using gmail.com> wrote:
>
>> Hi Jack,
>>
>> I would recommend using the first strategy, in which you create an
>> additional ID variable to distinguish independent samples nested within a
>> study. Just as a matter of coding, this is a better representation of the
>> structure of your data. You can always then simplify to get the data you'd
>> have from the other strategy (where you ignore the study/sample
>> distinction). But if you follow the second strategy, there's not an easy
>> way to add in the study and sample IDs without going back to recode.
>>
>> How you ultimately approach modeling the data is an empirical question.
>> With only two studies that have multiple samples, it is probably not
>> reasonable to include random effects at both the study level and the sample
>> level. But you could consider using either ~ 1 | studyID or ~ 1 | sampleID
>> (assuming that sample has a unique value for every unique sample). The
>> former assumes that the true effect for a given study is constant across
>> samples nested within that study. The latter assumes that the true effects
>> from samples in the same study are no more closely related than the true
>> effects from different studies.
>>
>> James
>>
>> On Mon, Jun 21, 2021 at 1:13 PM Jack Solomon <kj.jsolomon using gmail.com>
>> wrote:
>>
>>> Hello All,
>>>
>>> I have come across a couple of primary studies in my meta-analytic pool
>>> that have used independent samples of participants in them (e.g., high
>>> schoolers & middle schoolers).
>>>
>>> Question: I was wondering how exactly I should code these studies to
>>> account for their use of independent samples of participants?
>>>
>>> Should I create a new column ('sample') to distinguish between studies'
>>> samples (see below)? OR with just two such multi-sample studies, basically
>>> that is not worth it in which case the question becomes:
>>>
>>> Should I code each independent sample as an independent study (which
>>> ignores the correlation between true effect sizes from samples under each
>>> study)? see below.
>>>
>>> Thanks, Jack
>>>
>>> ***consider 'sampel' in coding:
>>> study sample es
>>> 1         1         .1
>>> 1         1         .2
>>> 1         2         .3
>>> 1         2         .4
>>> 2         1         .5
>>> 2         2         .6
>>> 3         1         .7
>>>
>>> ***ignore 'sample' in coding:
>>> study es
>>> 1         .1
>>> 1         .2
>>> 2         .3
>>> 2         .4
>>> 3         .5
>>> 4         .6
>>> 5         .7
>>>
>>>          [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-sig-meta-analysis mailing list
>>> R-sig-meta-analysis using r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis



More information about the R-sig-meta-analysis mailing list