[R-sig-ME] lmer fails when too many observations

[Ben Bolker]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

  Just a quick update on this.
  This 'failure' is coming because (1) the current version of lme4 has
too strict a test for convergence and gives what we think are
false-positive warnings, especially for large data sets; (2) you have
set options(warn=2) so that warnings get converted into errors.  Until
we get the issue fixed (which could take a while, as we haven't yet
given up on finding a more principled way than just increasing the
default tolerance a lot) you can either (a) convert your warnings back
to regular warnings (i.e. options(warn=1) or options(warn=0) or (b)
increase the tolerance level, e.g.

 control= lmerControl(check.conv.grad = .makeCC("warning", tol = 1e-2))

or

 control = lmerControl(check.conv.grad = "ignore")



On 15-03-09 09:52 PM, Asaf Weinstein wrote:
> Dear lmer community,
> 
> I am trying to run a simulation for a two-way random-effects model
> with unbalanced design (ie, unequal number of observations per
> cell) and no interaction. It's especially important for me to be
> able to run the lmer/blmer functions when the number of (column and
> row) random effects is large, say 100, and with possible replicates
> in each cell. The problem is that lmer() works with the full vector
> of observations, as opposed to working with the cell averages
> (which is a sufficient statistic), and the methods fails pretty
> quickly when there are replicates (because the response vector is
> too big, I suppose). I get the following error:
> 
> *Error in get("checkConv", lme4Env)(attr(opt, "derivs"), opt$par,
> ctrl = control$checkConv,  : * *  (converted from warning) Model
> failed to converge with max|grad| = 0.00244385 (tol = 0.002)*
> 
> Just to give an example: suppose there are R=100 row effects, C=100
> column effects, and 5 replicates in each cell. The vector of
> individual observations is of length 100^5 (lmer fails), while the
> vector of cell averages is of length 100^2 (a size which causes no
> problem for lmer). My question is whether there is a way to tell
> lmer() to work with the sufficient statistic (of course, the
> conditional covariance is no longer c*Identity, a fact which is
> used in the implementation of lmer (according to documentation) ).
> 
> Thank you very much and I hope I was clear!



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBAgAGBQJVALMaAAoJEOCV5YRblxUHnRwH/j8/MFNfYKgvgzbBBCdmfXOi
nfRGRpGWrFUM7MznFWFf4hdA0SFPUOTS9yZYxoGuyJHos1q5WeAyR9kdlqNQjdDx
lPV+zYy9gsj+uMl5XBVSurDrfJY+JXfnqtOzZECMbrZjNxHgTMnSxzMdRNVZAVm6
DaCAaNGJkeTI96k4jg7ucj1p/xGgk3deVW8NMY3bKMasd+dtFzTXqPDvEYovBRQT
sbQ7SavzjJY93QYHabaaDkpsD8N86qe5fJgOE9uVLNkvuOGVPELvwG8lCaC/kj7y
wdd4+HOum1AgeBUosCRfs+Ld41V4wBUySxFW9coLRUVwzQGfU8rOTjIpKs4iCpY=
=jitR
-----END PGP SIGNATURE-----