[R-meta] rma.glmm model selection

[Philippe Tadger]

Hello Wolfgang, colleagues

Thanks for the answer!

I've been reading the article  (Jackson et al., 2017) 
<https://doi.org/10.1002/sim.7588>were you explain such models and 
provide codes for SAS/Stata/R.

I did run the SAS code with the sham data, and collect all goodness of 
fit for models 2-6


	AIC 	BIC 	-2 x Likelihood 	Model Name 	effects 	Variance-covariance 
Matrix 	Other name
Model 4 	80.74 	80.25 	62.74 	modified Simmonds and Higgins model 	Fixed 	
	UM.FS
Model 2 	81.53 	81.05 	63.53 	Simmonds and Higgins model 	Fixed 	
	
Model 5 	91.19 	90.97 	83.19 	modified Simmonds and Higgins model 
Random 	CV 	UM.RS
Model 6 	93.18 	92.91 	83.18 	Van Houwelingen bivariate model 	Random 
UN 	CM.AL
Model 3 	93.76 	93.54 	85.76 	Simmonds and Higgins model 	Random 	CV 	


I understand that the likelihood measure can not be used if the 
distributions or models are not nested (models 6 & 7 or CM.AL and 
CM.EL), which is not the case as you point it out.
Do you consider the AIC values also not "meaningful" to choose between 
models?

Sorry if there's is any typo in the self made table (trying to unify all 
the names and features of each model)

Thanks in advance for you valuable time


On 12/09/2021 15:00, Viechtbauer, Wolfgang (SP) wrote:
> Hi Philippe,
>
> Good question. I doubt that a direct comparison of the likelihoods of these models (or information criteria) is meaningful though. For example, UM.FS and UM.RS differ both in terms of their fixed and random effects (since the whole point is to use either fixed or random study effects). CM.AL and CM.EL differ even more fundamentally, as they use other distributions.
>
> The sentence that you quote is true, but the devil is, as always, in the details. Here, I was thinking more in terms of: We have some specific model where we can swap in or out certain random effects. Once you start swapping in and out fixed and random effects at the same time and even switching distributions, then things get a lot more tricky.
>
> So, I really don't have any good suggestions at the moment.
>
> Best,
> Wolfgang
>
>> -----Original Message-----
>> From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces using r-project.org] On
>> Behalf Of Philippe Tadger
>> Sent: Saturday, 11 September, 2021 12:42
>> To: r-sig-meta-analysis using r-project.org
>> Subject: [R-meta] rma.glmm model selection
>>
>> Dear Meta community
>>
>> I would like to ask for guidance on  how to do model selection in
>> metafor::rma.glmm. I'm thinking specifically in the comparison between:
>> UM.FS, UM.RS, CM.AL, CM.EL methods. Is it possible to conduct a goodness
>> of fit or alternative selection methods between the 4 models?
>>
>> I saw the model selection here
>> https://www.metafor-
>> project.org/doku.php/tips:model_selection_with_glmulti_and_mumin
>> <https://www.metafor-
>> project.org/doku.php/tips:model_selection_with_glmulti_and_mumin>
>> for predictors that mention that is possible to do a similar approach
>> with rma.glmm. Also a specific phrase catch my attention: "one can also
>> consider model selection with respect to the random effects structure."
>>
>> Thanks in advance for your help and time
>>
>> --
>>
>> Kind regards/Saludos cordiales
>> *Philippe Tadger*
>> ORCID <https://orcid.org/0000-0002-1453-4105>, Reseach Gate
>> <https://www.researchgate.net/profile/Philippe-Tadger>
>> Phone/WhatsApp: +32498774742
-- 
Kind regards/Saludos cordiales
*Philippe Tadger*
ORCID <https://orcid.org/0000-0002-1453-4105>, Reseach Gate 
<https://www.researchgate.net/profile/Philippe-Tadger>
Phone/WhatsApp: +32498774742

	[[alternative HTML version deleted]]