[R-meta] Studies with independent samples of participants

[Dr. Gerta Rücker]

Dear Jack,

Is this distinction between different definitions of "control" also made 
in other studies? That is, are there throughout two (or more) types of 
control?

If so, this looks a bit like a network meta-analysis could also be 
appropriate.

Best,

Gerta

Am 21.06.2021 um 22:48 schrieb Jack Solomon:
> Much appreciated, James. There is one more complication in these two
> studies. They have used two comparison/control groups (as the definition of
> 'control' has been debated in the literature I'm meta-analyzing).
>
> Given that, in addition to the 'sample' column, should I create an
> additional column ('control') to distinguish between effect sizes (SMDs in
> this case) that have been obtained by comparing the treated groups to
> control 1 vs. control 2 (see below)?
>
> If yes, then, does the addition of a 'control' column call for the addition
> of a random effect for 'control' as well (again, something to be
> empirically tested)?
>
> Thanks again, Jack
>
> ***consider 'sample' & 'control' in coding:
> study sample es  control
> 1         1         .1      1
> 1         1         .2      2
> 1         2         .3      1
> 1         2         .4      2
> 2         1         .5      1
> 2         2         .6      2
> 3         1         .7      1
>
> On Mon, Jun 21, 2021 at 3:25 PM James Pustejovsky <jepusto using gmail.com> wrote:
>
>> Hi Jack,
>>
>> I would recommend using the first strategy, in which you create an
>> additional ID variable to distinguish independent samples nested within a
>> study. Just as a matter of coding, this is a better representation of the
>> structure of your data. You can always then simplify to get the data you'd
>> have from the other strategy (where you ignore the study/sample
>> distinction). But if you follow the second strategy, there's not an easy
>> way to add in the study and sample IDs without going back to recode.
>>
>> How you ultimately approach modeling the data is an empirical question.
>> With only two studies that have multiple samples, it is probably not
>> reasonable to include random effects at both the study level and the sample
>> level. But you could consider using either ~ 1 | studyID or ~ 1 | sampleID
>> (assuming that sample has a unique value for every unique sample). The
>> former assumes that the true effect for a given study is constant across
>> samples nested within that study. The latter assumes that the true effects
>> from samples in the same study are no more closely related than the true
>> effects from different studies.
>>
>> James
>>
>> On Mon, Jun 21, 2021 at 1:13 PM Jack Solomon <kj.jsolomon using gmail.com>
>> wrote:
>>
>>> Hello All,
>>>
>>> I have come across a couple of primary studies in my meta-analytic pool
>>> that have used independent samples of participants in them (e.g., high
>>> schoolers & middle schoolers).
>>>
>>> Question: I was wondering how exactly I should code these studies to
>>> account for their use of independent samples of participants?
>>>
>>> Should I create a new column ('sample') to distinguish between studies'
>>> samples (see below)? OR with just two such multi-sample studies, basically
>>> that is not worth it in which case the question becomes:
>>>
>>> Should I code each independent sample as an independent study (which
>>> ignores the correlation between true effect sizes from samples under each
>>> study)? see below.
>>>
>>> Thanks, Jack
>>>
>>> ***consider 'sampel' in coding:
>>> study sample es
>>> 1         1         .1
>>> 1         1         .2
>>> 1         2         .3
>>> 1         2         .4
>>> 2         1         .5
>>> 2         2         .6
>>> 3         1         .7
>>>
>>> ***ignore 'sample' in coding:
>>> study es
>>> 1         .1
>>> 1         .2
>>> 2         .3
>>> 2         .4
>>> 3         .5
>>> 4         .6
>>> 5         .7
>>>
>>>          [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> R-sig-meta-analysis mailing list
>>> R-sig-meta-analysis using r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis