[R-sig-ME] taking in account results of a gmml in despite, of error warning about memory?

[ONKELINX, Thierry]

Dear Glenda,

Your (0+spp|tree) random effect requires a 89 x 89 matrix , or 3916 (co)variances to be estimated. You don't have enough data to support that kind of model. Furthermore it will probably require large design matrices and thus lead to the out of memory errors you got. Maybe (1|tree) + (1|spp) is a doable random effects model that is still sensible. This is a cross effect of tree and spp. You even could consider (1|tree) + (1|spp) + (1|tree:spp). (1|tree) = effect of tree regardsless of spp. (1|spp) = effect of spp regardless of tree. (1|tree:spp) = combined effects of tree and spp.

Best regards,

Thierry


ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek / Research Institute for Nature and Forest
team Biometrie & Kwaliteitszorg / team Biometrics & Quality Assurance
Kliniekstraat 25
1070 Anderlecht
Belgium
Thierry.Onkelinx at inbo.be
www.inbo.be

To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of.
~ Sir Ronald Aylmer Fisher

The plural of anecdote is not data.
~ Roger Brinner

The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data.
~ John Tukey

-----Oorspronkelijk bericht-----
Van: glenda mendieta [mailto:glendamendieta at gmail.com] 
Verzonden: dinsdag 29 november 2011 22:31
CC: chris at trickysolutions.com.au; ONKELINX, Thierry
Onderwerp: Re: Re: [R-sig-ME] taking in account results of a gmml in despite, of error warning about memory?

Hi to everyone,
thanks very much to Chris, Thierry and Ben for your help, I really appreciated the feedback, here the follow up if you or anyone else were
interested:

*Chris*, I did try what u proposed, but instead of 4061 the error came up to 4095. When came to a smaller subset or removing interaction terms, or fixed effects it didn't run either (always showed the same error). 
Neither when only including RE, apparently this set up is impossible/to detailed to run.

*Thierry*, the pretty big covariance matrix is definitively an issue here, but the problem is that I indeed have different values of abundance of each epiphyte species (spp) ocurring/growing on each tree in each census and those are temporally correlated. I have set it up as:(1|tree/spp), which is species nested in each tree, but isn't that telling me that species occurring in one tree are not correlated to each other in time?, which wouldn't be the case my system (please see that I describe the system below).

*Ben*:
> Message: 4
> Date: Mon, 28 Nov 2011 16:40:50 +0000 (UTC)
> From: Ben Bolker <bbolker at gmail.com>
> To: r-sig-mixed-models at r-project.org
> Subject: Re: [R-sig-ME] taking in account results of a gmml in despite
> 	of	error warning about memory?
> Message-ID: <loom.20111128T173451-444 at post.gmane.org>
> Content-Type: text/plain; charset=us-ascii
>
> glenda mendieta <glendamendieta at ...> writes:
>
>> I have being trying to fit a glmm on binary and poisson data and when I
>> run this model, with poisson data, the error below shows up. But still
>> gives me results.
>
>   Be very careful.  It probably means that the results are left over
> from some previous run that you tried that did work.  An Error
> results should *not* give any result.  Try re-running in a clean
> R session, with just the minimal stuff you need loaded (see below).
>
True, an error doesn't give a result.
>> Does this means that those are only partial results
>> and shouldn't be taking into account, because the model didn't run fully?
>>
>> >  glmmab.FMv<-glmer(abundance~c.census*avail.surface*abundance.prev
>> +(0+spp|tree),
>> data=db.e_St, family=poisson(link=log))
>
>   Your random effect specification is not sensible (I think): it asks for an
> estimate of the variation of the species effect across trees, which
> is more or less impossible because every tree belongs only to a single
> species.  Did you mean (1|spp/tree) or (1|spp)+(1|tree) ?
>
The data is longitudinal and consists on values of abundance (counts) of 
89 vascular epiphyte species growing on different individuals of the 
same species of tree (~113 individuals) in a span of 10 years (the data 
was collected by means of 5 censuses, inspecting each tree and following 
the fate of all epiphyte individuals). See below str(data).

My aim was to know the rate of change in abundance of species in time 
and whether availability of surface plays a role in those changes (never 
mind "abundance.prev" term).
That specification of RE is far more detailed than what I am looking 
for, if I did understand you correctly when I assume that the pertain 
specification would be estimating a separate variance of trees for each 
level of species?.
I thought I was both including the variation of species across trees and 
controlling for their temporal correlation while using "trees", since 
those where inspected in each census. I have run models as you specified 
above: nested, as well as crossed, but I wasn't sure whether they 
depicted the set up of the data. Specially because with the nested model 
I thought the temporal correlation among species wasn't specified and 
then the crossed model shows a more than double-fold value of AIC than 
of the nested one (although both use up the same DF's).

>> I am using the latest version of R and R studio. As I have seen before
>> that some complicated models don't run at all if I had already many
>> other models as objects in the workspace, I did run this one with the
>> minimum use of memory (just the database as an object). I also read that
>> R is suppose to do not have memory problems any more, but I don't really
>> know how to expand the use of memory by R on my pc.
>
>    Take a look at the R for Windows FAQ entries on memory use.
>    Can you give the results of sessionInfo()?  If you can, you
> may need to switch to a 64-bit OS.

 > sessionInfo()
R version 2.13.2 (2011-09-30)
Platform: x86_64-pc-mingw32/x64 (64-bit)
>
>>  For what I observed
>> with mem.limits() it appears unlimited (NA), but then if I type
>> mem.limit(), 4061 shows up. Does that mean that I can not run those
>> models in my pc at al?.
>> Here, some more info in the data:
>>
>> Number of obs: 23407, groups:tree,89
>
>   This does not seem like a particularly huge dataset, so I'm
> a bit surprised you're running into trouble (with the exception
> of the weird RE specification ...) are your predictor variables
> all continuous?
>
In the case above, census is a factor, avail.surface is continuous. The 
other variable never mind. I could also set time as a continuous 
variable, but as I see it that wouldn't give me the rate of change.

>>
>> Thanks to anyone who can shed some light on this,
>>
>> Glenda Mendieta-Leiva
>> PhD candidate
>> University of Oldenburg

'data.frame':	36935 obs. of  8 variables:
  $ census        : Factor w/ 5 levels "1999-03-01","2002-07-01",..: 1 1 
1 1 1 1 1 1 1 1 ...
  $ tree          : Factor w/ 113 levels "35357","35373",..: 1 1 1 1 1 1 
1 1 1 1 ...
  $ spp           : Factor w/ 89 levels "AECTIL","ANAANG",..: 1 2 3 4 5 
6 7 8 9 10 ...
  $ abundance     : int  0 2 0 0 0 0 0 0 0 0 ...
  $ abs.pres      : Factor w/ 2 levels "0","1": 1 2 1 1 1 1 1 1 1 1 ...
  $ avail.surface : num  4.44 4.44 4.44 4.44 4.44 ...
  $ c.census      : num  10651 10651 10651 10651 10651 .


Thanks again,

Glenda Mendieta-Leiva