[R] Error: cannot allocate vector of size 1.2 Gb

[joris meys]

That's the file I needed, and the problem I expected. The factor you
specify is not a factor, but a numerical variable. Even more, if you
tabulate it, you have 2 times the values "1", "2" and "5", and all
other values only once. That's what gives you the error.

p is an internal variable of the function specaccum. It's the number
of columns in a  dataframe containing only locations with a certain
value for the specified factor. If there is only one location with
that factor level, you get only one row and thus a vector. The
function ncol() gives result NULL if you try it on a vector,  so p is
NULL, and has zero length.

You can for example convert HM_sprem to a factor, indicating "low" and
"high" values.

PoEnv$test <-factor(ifelse(PoEnv$HM_sprem<2.5,"low","high"),levels=c("low","high"),ordered=T)
poacc2 <- accumcomp(PoCom, y=PoEnv, factor="test", method="exact")

works like a charm for me.

In the future, check carefully which type of arguments are asked for,
and use the function str() to check if they really are what you think
they are.

Kind regards
Joris


---------- Forwarded message ----------
From: romunov <romunov at gmail.com>
Date: Mon, Oct 12, 2009 at 3:14 PM
Subject: Re: [R] Error: cannot allocate vector of size 1.2 Gb
To: joris meys <jorismeys at gmail.com>


I hope this workspace file attached is what you were looking for.

Cheers,
Roman


On Mon, Oct 12, 2009 at 2:07 PM, joris meys <jorismeys at gmail.com> wrote:
>
> It's easier if you just load the data in R, save the workspace and
> send that one. I really don't have time to read in csv data and find
> my way trough it, sorry. It also makes it possible for me to find
> errors due to the wrong data format (factors that are not defined as
> factor and the likes).
>
> Kind regards
> Joris
>
> On Mon, Oct 12, 2009 at 2:04 PM, romunov <romunov at gmail.com> wrote:
> > Hi again,
> >
> > thank you for your time, I really appreciate you taking time dealing with my
> > problem. I am sending you my dataset (community (zdruzbe_analiza.csv) and
> > environmental (okoljski_analiza.csv) factors) in case you want to try it out
> > for yourself.
> >
> > Cheers,
> > Roman
> >
> >
> >
> > On Mon, Oct 12, 2009 at 2:02 PM, romunov <romunov at gmail.com> wrote:
> >>
> >> Hi Joris,
> >>
> >> thanks for spotting that one. This little mistake has gotten in when I was
> >> trying desperate things with the analysis (factor1 is used in
> >> diversitycomp). Nevertheless, here is the result:
> >>
> >> > poacc2 <- accumcomp(PoCom, y=PoEnv, factor="HM_sprem", method="exact")
> >> Error in if (p == 1) { : argument is of length zero
> >>
> >> And the traceback():
> >>
> >> > traceback()
> >> 3: specaccum(x, method = method, permutations = permutations, conditioned
> >> = conditioned,
> >>        gamma = gamma, ...)
> >> 2: accumresult(x, y, factor, level = levels[i], scale = scale, method =
> >> method,
> >>        permutations = permutations, conditioned = conditioned, gamma =
> >> gamma)
> >> 1: accumcomp(PoCom, y = PoEnv, factor = "HM_sprem", method = "exact")
> >>
> >> Best wishes,
> >> Roman
> >>
> >>
> >> On Mon, Oct 12, 2009 at 1:56 PM, joris meys <jorismeys at gmail.com> wrote:
> >>>
> >>> Hi Roman,
> >>>
> >>> that throws a different light on the problem. It goes wrong from the
> >>> start, so it has little to do with the bootstrap or jackknife
> >>> procedures. R.huge won't help you either.
> >>>
> >>> Likely your error comes from the fact that "factor1" is not an
> >>> argument of the function accumcomp. the argument is "factor". As R
> >>> doesn't find this, it'll try to tabulate the complete environmental
> >>> dataset, and this gives the memory overflow.
> >>>
> >>> Try :
> >>>
> >>> poacc2 <- accumcomp(PoCom, y=PoEnv, factor="HM_sprem", method="exact")
> >>>
> >>> That should work. I can't try it out without dataset off course. If it
> >>> doesn't work, post the traceback again, I'll take another look.
> >>>
> >>> Kind regards
> >>> Joris
> >>>
> >>> On Mon, Oct 12, 2009 at 1:07 PM, romunov <romunov at gmail.com> wrote:
> >>> > Hello joris,
> >>> >
> >>> > this is the traceback() output. Hopefully you can make some sense out
> >>> > of it.
> >>> > Thank you for the tips as well (R.huge looks promising)!
> >>> >
> >>> >> traceback()
> >>> > 7: vector("integer", length)
> >>> > 6: integer(nbins)
> >>> > 5: tabulate(bin, pd)
> >>> > 4: as.vector(data)
> >>> > 3: array(tabulate(bin, pd), dims, dimnames = dn)
> >>> > 2: table(y[, factor])
> >>> > 1: accumcomp(PoCom, y = PoEnv, factor1 = "HM_sprem", method = "exact")
> >>> >
> >>> > Cheers,
> >>> > Roman
> >>> >
> >>> >
> >>> >
> >>> > On Mon, Oct 12, 2009 at 12:52 PM, joris meys <jorismeys at gmail.com>
> >>> > wrote:
> >>> >>
> >>> >> Dear Roman,
> >>> >>
> >>> >> could you give us the trace given by traceback() ? I suspect the error
> >>> >> is resulting from the permutations and/or jackknife procedure in the
> >>> >> underlying functions specaccum and specpool.
> >>> >>
> >>> >> You can take a look at the package R.huge, but that one is deprecated
> >>> >> already. There are other packages around too, but I have no experience
> >>> >> with them. You find some more tips here :
> >>> >> http://www.matthewckeller.com/html/memory.html
> >>> >>
> >>> >> This should give you a place to start looking.
> >>> >> Kind regards
> >>> >> Joris
> >>> >>
> >>> >> On Mon, Oct 12, 2009 at 11:39 AM, romunov <romunov at gmail.com> wrote:
> >>> >> > Dear List,
> >>> >> >
> >>> >> > today I turn to you with a next problem. I'm trying to compare
> >>> >> > species
> >>> >> > richness between various datasets (locations) using species
> >>> >> > accumulation
> >>> >> > curves (Chapter 4, page 54 in Tree diversity
> >>> >> >
> >>> >> >
> >>> >> > analysis<http://www.worldagroforestry.org/treesandmarkets/tree_diversity_analysis.asp>by
> >>> >> > Kindt & Coe). To accomplish this I'm using package
> >>> >> > BiodiversityR. My data is comprised of species community (PoCom) (10
> >>> >> > locations with 83 species) and environmental factors (PoEnv) (10
> >>> >> > locations
> >>> >> > with 17 factors).
> >>> >> > In attempt to calculate the function (accumcomp) I receive the
> >>> >> > following
> >>> >> > error. I can not imagine how a 10x83+10x17 matrix can grow to a GB
> >>> >> > or
> >>> >> > more.
> >>> >> > Unless I'm missing something? How can I combat this?
> >>> >> >
> >>> >> >> poacc2 <- accumcomp(PoCom, y=PoEnv, factor1="HM_sprem",
> >>> >> >> method="exact")
> >>> >> > Error: cannot allocate vector of size 1.2 Gb
> >>> >> > In addition: Warning messages:
> >>> >> > 1: In vector("integer", length) :
> >>> >> >  Reached total allocation of 1023Mb: see help(memory.size)
> >>> >> > 2: In vector("integer", length) :
> >>> >> >  Reached total allocation of 1023Mb: see help(memory.size)
> >>> >> > 3: In vector("integer", length) :
> >>> >> >  Reached total allocation of 1023Mb: see help(memory.size)
> >>> >> > 4: In vector("integer", length) :
> >>> >> >  Reached total allocation of 1023Mb: see help(memory.size)
> >>> >> >
> >>> >> >
> >>> >> > Cheers,
> >>> >> > Roman
> >>> >> >
> >>> >> >        [[alternative HTML version deleted]]
> >>> >> >
> >>> >> > ______________________________________________
> >>> >> > R-help at r-project.org mailing list
> >>> >> > https://stat.ethz.ch/mailman/listinfo/r-help
> >>> >> > PLEASE do read the posting guide
> >>> >> > http://www.R-project.org/posting-guide.html
> >>> >> > and provide commented, minimal, self-contained, reproducible code.
> >>> >> >
> >>> >
> >>> >
> >>
> >
> >