[R-meta] errors returned by rma() and rma.mv() when fitting a large dataset

[Viechtbauer, Wolfgang (NP)]

Dear Yefeng,

The problem is that quite a bit of code in metafor works with k*k matrices, where k is the number of estimates. A matrix of that size can quickly get very large, as you can tell.

To fit a RE model, you can use rma.mv() with sparse=TRUE. This will avoid those large matrices. However, trimfill() and selmodel() won't work with rma.mv objects, even if they are just 'standard' RE models.

I *think* (but would have to double-check very carefully) that selmodel() actually doesn't make use of k*k matrices. If so, you could take the rma.mv results, stuff them into an object that has the structure of a rma.uni object, and then call selmodel() on that object. Not very elegant, but this could be a solution. But trimfill() directly calls rma.uni() for model fitting, so this trick wouldn't work. In principle, one could spin a trimfill() version that avoids calling rma.uni() but this would take some work.

Maybe you could try the 'meta' package and its trimfill() function?

Best,
Wolfgang

> -----Original Message-----
> From: R-sig-meta-analysis <r-sig-meta-analysis-bounces using r-project.org> On Behalf
> Of Yefeng Yang via R-sig-meta-analysis
> Sent: Wednesday, May 22, 2024 11:19
> To: r-sig-meta-analysis using r-project.org
> Cc: Yefeng Yang <yefeng.yang1 using unsw.edu.au>
> Subject: [R-meta] errors returned by rma() and rma.mv() when fitting a large
> dataset
>
> Dear community,
>
> I am trying to test publication bias using trim-and-fill and selection model in
> metafor package. When I ran the RE model with my dataset, it returned my the
> following error:
>
> Error: cannot allocate vector of size 33.8 Gb
>
> I tried both rma() and rma.mv() (the later also can be used to fit RE model).
>
> I think this was caused by the large number of data points in my dataset. My
> dataset contains 67,393 rows (or more precisely, 67,393 paired effect size
> estimates and sampling variance).
>
> Is there any solution workaround that allows me to  fit a RE model with a large
> number of effect sizes, and perform the following trim-and-fill and selection
> model?
>
> Very much appreciate your comments.
>
> Best regards,
> Yefeng