[R-sig-ME] GAMM4 CHOLMOD error

[Simon Wood]

This is probably a gamm4 limitation. The computation of the posterior 
cov matrix for the model coefficients is inefficient when there are only 
a few levels of the random effect (which I guess is the case for 
`country'?). You can fit exactly the same model more efficiently using

bam(log(kg_day) ~ s(year) + s(country,bs="re"),weights=1/fishing_day,
data=temp, method="ML")

see the random.effects help file in mgcv for details.

(bam is large dataset version of gam. gam could fit the same model, but 
might be slow/memory intensive for 161000 data)

best,
Simon

On 07/09/11 16:04, Chato Osio wrote:
> Dear List,
>
> I am trying to fit a model in gamm4 but keep running into errors I
> can't seem to solve.
>
> My model simplified model is:
>   gamm4(log(kg_day) ~ s(year) , random = ~(1 | country), weights =
> 1/fishing_day, data = temp, REML = F)
>
> The error I receive  is:
>
> CHOLMOD error:
> Error in crossprod(root.phi %*% Zt) :
>   Cholmod error 'problem too large' at file:../Core/cholmod_aat.c, line 174
>
> The error from what I understand comes from the package MATRIX. Is
> this a bug or is it an intrinsic limitation with a "large" dataset? My
> dataset is
>
> dim(temp)
> [1] 161301      7
>
> summary(temp2)
>        year          kg_day              HP          country
>   Min.     : 1868   Min.   :    0.2   Min.   :    0.0   ES  :63084
>   1st Qu.: 1996   1st Qu.:   75.2   1st Qu.:  210.5   FR  :86825
>   Median :1999   Median :  160.7   Median :  423.6   ITA :11285
>   Mean   :1997   Mean   :  246.9   Mean   :  366.5   JUGO:   74
>   3rd Qu.:2003   3rd Qu.:  347.9   3rd Qu.:  423.6   TUNI:   33
>   Max.   : 2009   Max.   :11076.1   Max.   : 1200.0
>                                    NA's   :15275.0
>       area            fishing_day                type_trawler
>   Length:161301      Min.   :   0.300   motor          :161269
>   Class :character   1st Qu.:   1.000   pair_motor     :     3
>   Mode  :character   Median :   1.000   pairsailtrawler:    15
>                      Mean   :   2.249   steamtrawler   :    14
>                      3rd Qu.:   1.000
>                      Max.   :7074.000
>
>
> If I run on the same data an identical lmer model without a smoother
> on year I have no problems.
> lmer( log(kg_day)~ year+(1|country), weights=1/fishing_day,
> data=temp2,  REML = F)
>
>> summary(allmer)
> Linear mixed model fit by maximum likelihood
> Formula: log(kg_day) ~ year + (1 | country)
>     Data: temp
>      AIC    BIC  logLik deviance REMLdev
>   464720 464760 -232356   464712  464726
> Random effects:
>   Groups   Name        Variance Std.Dev.
>   country  (Intercept) 1.0744   1.0365
>   Residual             1.0551   1.0272
> Number of obs: 161301, groups: country, 5
>
> Fixed effects:
>                Estimate Std. Error t value
> (Intercept) -2.3843311  0.6885985  -3.463
> year         0.0039580  0.0002291  17.275
>
> Correlation of Fixed Effects:
>       (Intr)
> year -0.660
>
>
>
>> sessionInfo()
> R Under development (unstable) (2011-09-04 r56935)
> Platform: x86_64-unknown-linux-gnu (64-bit)
>
> locale:
>   [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>   [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
>   [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
>   [7] LC_PAPER=C                 LC_NAME=C
>   [9] LC_ADDRESS=C               LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
> other attached packages:
> [1] gamm4_0.1-3        mgcv_1.7-6         lme4_0.999375-41   Matrix_0.9996875-3
> [5] lattice_0.19-33
>
> loaded via a namespace (and not attached):
> [1] grid_2.14.0   nlme_3.1-102  stats4_2.14.0 tools_2.14.0
>
> Thanks for any suggestion/help
>
> Chato
>
>


-- 
Simon Wood, Mathematical Science, University of Bath BA2 7AY UK
+44 (0)1225 386603               http://people.bath.ac.uk/sw283