[R-meta] min sample size for three random variables in meta-analysis

[Viechtbauer, Wolfgang (NP)]

Dear Jose,

With this little data, you are not going to get accurate estimates of the variance components (in fact, the model may even be overparameterized, but I cannot easily check this without a fully reproducible example). Hence, I wouldn't put much trust into the findings.

In general, I am not aware of any useful rules though that one could use to decide on an appropriate model complexity in this context. In the multilevel literature, one sometimes finds suggestions that there should be at least a certain number (e.g., 10) of upper-level units (e.g., schools) before one can consider using random effects at that level. But these rules are typically overly simplistic and have not been examined in the meta-analytic context as far as I know.

You might have to stick to a simpler (e.g., standard random-effects model) for this analysis and note that phylogenetic relatedness could not be accounted for due to the small sample size.

Best,
Wolfgang

>-----Original Message-----
>From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>Behalf Of Jose Valdebenito
>Sent: Friday, 22 July, 2022 16:50
>To: Michael Dewey
>Cc: r-sig-meta-analysis using r-project.org
>Subject: Re: [R-meta] min sample size for three random variables in meta-analysis
>
>Dear Michael,
>Thanks for your reply. By sample size I mean small number of primary
>studies. k means number of effect sizes.
>In the example below there were 10 effect sizes, 7 studies, 9 species
>ID, and 9 species associated with a vcv value.
>
>> head(df, 10)
>species_2 | spp | study.ID | L_d | L_v
>Grus_americana | Grus_americana | p.b.4 | 0.000 | 0.135
>Loxia_leucoptera | Loxia_leucoptera | p.b.15 | 1.057 | 0.068
>Centrocercus_urophasianus | Centrocercus_urophasianus | p.b.60 | 0.384 | 0.021
>Pica_nuttalli | Pica_nuttalli | p.b.8 | -2.042 | 0.037
>Acrocephalus_paludicola | Acrocephalus_paludicola | 17 | 0.000 | 0.068
>Acrocephalus_paludicola | Acrocephalus_paludicola | 17 | -4.978 | 0.334
>Lepidothrix_coronata | Lepidothrix_coronata | 20 | -2.043 | 0.088
>Erithacus_rubecula | Erithacus_rubecula | 35 | -1.723 | 0.401
>Fringilla_coelebs | Fringilla_coelebs | 35 | -2.697 | 0.111
>Parus_caeruleus | Parus_caeruleus | 35 | 4.862 | 0.559
>
>Thanks,
>Jose
>On Fri, 22 Jul 2022 at 08:19, Michael Dewey <lists using dewey.myzen.co.uk> wrote:
>>
>> Dear Jose
>>
>> Unfortunately this is a plain text list and your post in HTML has got
>> scrambled so as to be almost unreadable.
>>
>> Just to clarify, when you say small sample size do you mean a small
>> number of primary studies or that individual studies were based on small
>> numbers? Is k the number of primary studies?
>>
>> Michael
>>
>> On 21/07/2022 15:42, Jose Valdebenito wrote:
>> > Hi,
>> >
>> > I am conducting a (multi-level) meta-analysis with some sample sizes that
>> > are a bit small.
>> >
>> > I was wondering if there is a kind of rule of thumb regarding minimum
>> > sample size number, and the maximum number of random variables I can add to
>> > the models.
>> >
>> > I would like to add 3 (as suggested in Cinar et al. Methods Ecol Evol.
>> > 2022;13:383–95), but I am afraid that might incur in overparameterization
>> > if I have, say k=10?
>> >
>> > These are the random variables:
>> > -Phylogenetic relatedness
>> > -Study ID
>> > -Species ID
>> >
>> > Also, would it be any different in a meta-regression?
>> >
>> > Here I paste the model and dataset so you can see:
>> >
>> > m2 <- rma.mv(L_d, L_v,
>> >                          data = df,
>> >                           random = list(~1|species_2,
>> >                                                 ~1|study.ID,
>> >                                                 ~1|spp),
>> >                            R = list(species_2 = vcv), digits = 3)
>> >
>> >> head(df, 10)
>> > species_2                                 spp
>> >             study.ID         L_d         L_v
>> > Grus_americana                       Grus_americana
>> > p.b.4             0.000       0.135
>> > Loxia_leucoptera                      Loxia_leucoptera
>> > p.b.15          1.057        0.068
>> > Centrocercus_urophasianus    Centrocercus_urophasianus   p.b.60
>> > 0.384        0.021
>> > Pica_nuttalli                              Pica_nuttalli
>> >          p.b.8            -2.042      0.037
>> > Acrocephalus_paludicola          Acrocephalus_paludicola       17
>> >      0.000      0.068
>> > Acrocephalus_paludicola          Acrocephalus_paludicola       17
>> >        -4.978      0.334
>> > Lepidothrix_coronata                Lepidothrix_coronata              20
>> >               -2.043     0.088
>> > Erithacus_rubecula                   Erithacus_rubecula                35
>> >                -1.723      0.401
>> > Fringilla_coelebs                       Fringilla_coelebs
>> >   35                 -2.697      0.111
>> > Parus_caeruleus                       Parus_caeruleus                    35
>> >                 4.862      0.559
>> >
>> > Thanks in advance,
>> > Joe