[R-meta] stepadj argument ignored in an rma.mv model

[Viechtbauer, Wolfgang (SP)]

Well, that fails spectacularly. Not a single successful fit across all 20 sigma^2 values (all ll values are NA). This is a bit surprising, since the actual estimate of sigma^2 is 0 and even for this value the profiling fails. I can't tell if this is a problem with the profile() function or due to a model that is so overparameterized that all bets are off. Would you be willing to send me the data so I can do some more testing to figure out what is going on?

Best,
Wolfgang

-----Original Message-----
From: Danka Puric [mailto:djaguard using gmail.com] 
Sent: Tuesday, 07 May, 2019 23:12
To: Viechtbauer, Wolfgang (SP)
Cc: r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] stepadj argument ignored in an rma.mv model

Dear Wolfgang,

here is the output of the dput(sav):

> dput(sav)
structure(list(sigma2 = c(0, 0.00526315789473684, 0.0105263157894737, 
0.0157894736842105, 0.0210526315789474, 0.0263157894736842, 0.0315789473684211, 
0.0368421052631579, 0.0421052631578947, 0.0473684210526316, 0.0526315789473684, 
0.0578947368421053, 0.0631578947368421, 0.068421052631579, 0.0736842105263158, 
0.0789473684210526, 0.0842105263157895, 0.0894736842105263, 0.0947368421052632, 
0.1), ll = c(NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_), beta = structure(list(intrcpt = c(NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_)), class = "data.frame", row.names = c(NA, 
-20L)), ci.lb = structure(list(intrcpt = c(NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_)), class = "data.frame", row.names = c(NA, 
-20L)), ci.ub = structure(list(intrcpt = c(NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_)), class = "data.frame", row.names = c(NA, 
-20L)), comps = 1, ylim = c(Inf, -Inf), method = "REML", vc = 0, 
    maxll = structure(-26.6009639211932, nall = 70L, nobs = 69L, df = 6, class = "logLik"), 
    xlab = expression(paste(sigma^2, " Value")), title = expression(
        paste("Profile Plot for ", sigma^2))), class = "profile.rma")

And yes, we realize that the number of effect sizes is rather small. We were only able to identify 14 papers with 16 independent studies, so we're trying to maximize the information we can extract by fitting a model with nested random effects. However, we understand it might ultimately be necessary to switch to another approach for dealing with effect size dependence.

Thanks for all your help!
Danka  

On Tue, May 7, 2019 at 9:51 PM Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer using maastrichtuniversity.nl> wrote:
Please do:

sav <- profile(es.re.mv, sigma2=1, plot=FALSE)

and then:

dput(sav)

and paste the output here.

This aside, you are fitting a rather complex model with 'only' 70 estimates. I cannot tell you whether this (and the model) makes sense.

Best,
Wolfgang

-----Original Message-----
From: Danka Puric [mailto:djaguard using gmail.com] 
Sent: Tuesday, 07 May, 2019 20:59
To: Viechtbauer, Wolfgang (SP)
Cc: r-sig-meta-analysis using r-project.org
Subject: Re: [R-meta] stepadj argument ignored in an rma.mv model

Dear Wolfgang,

yes, you are correct, I was using Rstudio. Profiling in basic R resulted in this message:
Profiling sigma2 = 1 
...
|=========================================================================================================================| 100%
Error in plot.window(...) : need finite 'ylim' values
In addition: Warning messages:
1: In min(x) : no non-missing arguments to min; returning Inf
2: In max(x) : no non-missing arguments to max; returning -Inf

Again, this is similar to what I get when plotting over single variance components.

I have installed the developer version of metafor, fitted the model again and plotted the variance components, but unfortunately I haven't noticed anything different (the model vaues are the same as well as the profile() warning messages).

Finally, here is the model I'm fitting as well as the full output:

> es.re.mv <- rma.mv(ES_corrected, SV, random = list(~ 1 | MA_data$IDeffect,
+                                                      ~ MA_data$DV | MA_data$IDsubsample,
+                                                      ~ MA_data$IDsubsample | MA_data$IDstudy), 
+                      struct="CS", data=MA_data, control=list(optimizer="optim", optmethod="Nelder-Mead"))
> es.re.mv

Multivariate Meta-Analysis Model (k = 70; method: REML)

Variance Components: 

            estim    sqrt  nlvls  fixed            factor
sigma^2    0.0000  0.0000     70     no  MA_data$IDeffect

outer factor: MA_data$IDsubsample (nlvls = 55)
inner factor: MA_data$DV          (nlvls = 5)

            estim    sqrt  fixed
tau^2      0.0843  0.2903     no
rho        1.0000             no

outer factor: MA_data$IDstudy     (nlvls = 16)
inner factor: MA_data$IDsubsample (nlvls = 55)

            estim    sqrt  fixed
gamma^2    0.0000  0.0002     no
phi        1.0000             no

Test for Heterogeneity: 
Q(df = 69) = 312.8672, p-val < .0001

Another huge thank you!
Danka