[R-sig-ME] How does blme carry out estimation and inference.

[Vincent Dorie]

Hi Gopal,

I apologize if I'm a little too informal in my answers - it's been a long
time since I've thought about these things.

1. For generalized linear models, beta isn't integrated out. The objective
function is a numerical approximation to the marginal likelihood - i.e.
with theta integrated - and is a function of sigma_theta and beta.
Estimation is accomplished by taking that objective function on the log
scale (deviance, actually) and adding to it the prior contribution.
2. It's not, not really. summary for glmms will return a normal
approximation based on the Hessian of the objective at its extremum, which
after fitting will include the prior's contribution. It's correct-ish*,
however people who are interested in posterior intervals are encouraged to
refit their model in a full Bayesian setting. blme was designed to rapidly
prototype parametric models and sits between lme4 and full Bayes - it can
avoid the boundary problem but is relatively fast.

Best,
Vince

*With a large sample size, the contribution of the prior vanishes and the
asymptotics hold. However, with a large sample size the boundary problem
should also go away. So for when blme might be useful in penalizing the
covariance of the random effects, there's pretty good reason to doubt the
uncertainty estimates are accurate. Technically one might be able to split
the difference by analyzing how much information the covariance parameters
contain about the fixed effects, but I've never looked into that myself.
However, if you just want to incorporate prior information on the fixed
effects or fix the residual standard deviation, the results should be
correct.

On Sat, Mar 4, 2023 at 3:40 PM Kotecha, Gopal <gkotecha using g.harvard.edu>
wrote:

> Dear all,
>
> I have two questions about blme.  Apologies if any of this has been
> asked before.
>
> 1. How does estimation occur?
> 2. How does inference occur for fixed effects parameters? Specifically
> when you use the “summary” method on a bglmer object, how are standard
> errors estimated?
>
> I have written up some more details below, and also pasted a LaTeX image
> here for easier reading:  https://imgur.com/a/syL1LWx
>
> I am using the default settings and the following logistic model:
>
> P(Y_{ij}=1|\beta, X_{ij},T_{ij},\theta_j) = F(X^T_{ij}  \beta + \theta_j)
> individuals i, groups j, where scalar \theta_j is distributed
> Normal(0,\sigma_\theta), outcome Y_{ij} is binary, X_{ij} is a vector of
> binary covariates with corresponding fixed effect parameter vector \beta.
> F(t) =(1+\exp(-t))^{-1}.
>
> My understanding is the following:
>
>   - BLME uses a penalized log likelihood approach, letting us place a
> prior on $\sigma_\theta$ to stabilize the inference and prevent zero
> values.  We can also place a prior on $\beta$
>
> - Inference is first carried out on the marginal log posterior density
> l(\sigma_\theta|Y_{ij}) where \beta and \theta_j are both integrated out
> to find the MAP estimate \hat\sigma_\theta.  This is then substituted
> into the original likelihood l(\hat\sigma_\theta,\theta|Y_{ij}) which is
> maximized to get \hat\beta. This is as far as I can tell from Chung
> et.a. 2013.  Is this correct?
>
> - How does blme estimate the standard errors of fixed effect estimates
> \hat\beta (i.e. what does summary on a model object produce)?  I can't
> find anything relating to this in the documentation.  Am I correct in
> thinking the marginal likelihood with the MAP estimate of
> \hat\sigma_\theta plugged in is profiled (i.e. a profile likelihood
> approach on l(\theta,\hat\sigma_\theta|Y_{ij}))?
>
> Any pointers, tips or references I have missed would be super useful!
>
> Thanks in advance!
> Gopal
>
> Refs:
>
> Chung, Yeojin, et al. "A nondegenerate penalized likelihood estimator
> for variance parameters in multilevel models." Psychometrika 78 (2013):
> 685-709.
> Bates, D., Mächler, M., Bolker, B., $\&$ Walker, S. (2015). Fitting
> Linear Mixed-Effects Models Using lme4. Journal of Statistical Software,
> 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01
>
> _______________________________________________
> R-sig-mixed-models using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
>

	[[alternative HTML version deleted]]