[R] R.matlab memory use

[Henrik Bengtsson]

Hi,

I am using Octave; what does that save options do, more specifically,
is compression taking place when saving that file?

If compression is done, then the Rcompression package is utilized by
R.matlab (otherwise not).  BTW, you don't have to load Rcompression
explicitly; R.matlab will do it for you if needed.  So, if you start a
fresh R session and load R.matlab and then try to load your package,
is Rcompression loaded?  If so, what version Rcompression do you have
installed, i.e. what does

sessionInfo()

report afterward?   Duncan TL did Rcompression updates addressing
memory usage about a year ago (I think) and it might be that you are
using an older version of it.  You should also update R.matlab et al,
because your using old versions (though I don't think that is the
cause here).

If Rcompression is the cause here, then it also make sense that you
don't experience the memory hog when reading a text file (which is
never compressed).  You could also see if there is an option in Octave
that safes to binary format but without compression.  I know Matlab
has such options.

/Henrik
(author of R.matlab)




On Mon, Dec 20, 2010 at 7:11 AM, Stefano Ghirlanda
<dr.ghirlanda at gmail.com> wrote:
> Hi Ben,
> Thanks for your reply. My data structure is about 20000 x 2000 so one
> order of magnitude the one you tried. I have no problem saving and
> reading smaller data structures (even large ones, just not his large)
> between octave and R using octave's "save -7" (which saves MATLAB v5
> files) and R.matlab's readMat. And I can save in text format in octave
> and read in R using read.octave (from package foreign) so it's not a
> big deal. I was just surprised that R.matlab needed more memory than I
> have (I have 3GB on this machine).
>
> Thanks,
> Stefano
>
> On Sun, Dec 19, 2010 at 10:54 PM, Ben Bolker <bbolker at gmail.com> wrote:
>> Stefano Ghirlanda <dr.ghirlanda <at> gmail.com> writes:
>>
>>> I am trying to load into R a MATLAB format file (actually, as saved by
>>> octave). The file is about 300kB but R complains with a memory
>>> allocation error:
>>>
>>> > library(Rcompression)
>>> > library(R.matlab)
>>> Loading required package: R.oo
>>> Loading required package: R.methodsS3
>>> R.methodsS3 v1.2.0 (2010-03-13) successfully loaded. See ?R.methodsS3 for
>>> help.
>>> R.oo v1.7.2 (2010-04-13) successfully loaded. See ?R.oo for help.
>>> R.matlab v1.3.1 (2010-04-20) successfully loaded. See ?R.matlab for help.
>>> > f <- readMat("freq.mat")
>>> Error: cannot allocate vector of size 296.5 Mb
>>>
>>> On the other hand, if I save the same data in ascii format (from
>>> octave: "save -text"), resulting in a 75MB file, then I can load it
>>> without problems with the read.octave() function from package foreign.
>>> Is this a known issue or am I doing something wrong? My R version is:
>>
>>  This is not a package I'm particularly familiar with, but:
>>
>>  what commands did you use to save the file in octave?  Based on
>> 'help save' I think that 'save' by default would get you an octave
>> format file ... you might have to do some careful reading in
>> ?readMat (in R) and 'help save' (in octave) to figure out the
>> correspondence between octave/MATLAB and R/MATLAB.
>>   If possible, try saving a small file and see if it works; if
>> you still don't know what's going on, post that file somewhere for
>> people to try.
>>
>>  I was able to
>>
>> save -6 "save.mat" in octave and
>> readMat("save.mat") in R successfully,
>> saving a vector of integers from 1 to 1 million (which
>> took about 7.7 Mb)
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>
>
> --
> Stefano Ghirlanda
> www.intercult.su.se/~stefano - drghirlanda.wordpress.com
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>