[R-meta] Studies with independent samples of participants

[Jack Solomon]

Much appreciated, James. There is one more complication in these two
studies. They have used two comparison/control groups (as the definition of
'control' has been debated in the literature I'm meta-analyzing).

Given that, in addition to the 'sample' column, should I create an
additional column ('control') to distinguish between effect sizes (SMDs in
this case) that have been obtained by comparing the treated groups to
control 1 vs. control 2 (see below)?

If yes, then, does the addition of a 'control' column call for the addition
of a random effect for 'control' as well (again, something to be
empirically tested)?

Thanks again, Jack

***consider 'sample' & 'control' in coding:
study sample es  control
1         1         .1      1
1         1         .2      2
1         2         .3      1
1         2         .4      2
2         1         .5      1
2         2         .6      2
3         1         .7      1

On Mon, Jun 21, 2021 at 3:25 PM James Pustejovsky <jepusto using gmail.com> wrote:

> Hi Jack,
>
> I would recommend using the first strategy, in which you create an
> additional ID variable to distinguish independent samples nested within a
> study. Just as a matter of coding, this is a better representation of the
> structure of your data. You can always then simplify to get the data you'd
> have from the other strategy (where you ignore the study/sample
> distinction). But if you follow the second strategy, there's not an easy
> way to add in the study and sample IDs without going back to recode.
>
> How you ultimately approach modeling the data is an empirical question.
> With only two studies that have multiple samples, it is probably not
> reasonable to include random effects at both the study level and the sample
> level. But you could consider using either ~ 1 | studyID or ~ 1 | sampleID
> (assuming that sample has a unique value for every unique sample). The
> former assumes that the true effect for a given study is constant across
> samples nested within that study. The latter assumes that the true effects
> from samples in the same study are no more closely related than the true
> effects from different studies.
>
> James
>
> On Mon, Jun 21, 2021 at 1:13 PM Jack Solomon <kj.jsolomon using gmail.com>
> wrote:
>
>> Hello All,
>>
>> I have come across a couple of primary studies in my meta-analytic pool
>> that have used independent samples of participants in them (e.g., high
>> schoolers & middle schoolers).
>>
>> Question: I was wondering how exactly I should code these studies to
>> account for their use of independent samples of participants?
>>
>> Should I create a new column ('sample') to distinguish between studies'
>> samples (see below)? OR with just two such multi-sample studies, basically
>> that is not worth it in which case the question becomes:
>>
>> Should I code each independent sample as an independent study (which
>> ignores the correlation between true effect sizes from samples under each
>> study)? see below.
>>
>> Thanks, Jack
>>
>> ***consider 'sampel' in coding:
>> study sample es
>> 1         1         .1
>> 1         1         .2
>> 1         2         .3
>> 1         2         .4
>> 2         1         .5
>> 2         2         .6
>> 3         1         .7
>>
>> ***ignore 'sample' in coding:
>> study es
>> 1         .1
>> 1         .2
>> 2         .3
>> 2         .4
>> 3         .5
>> 4         .6
>> 5         .7
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-meta-analysis mailing list
>> R-sig-meta-analysis using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>>
>

	[[alternative HTML version deleted]]