[R] R.matlab memory use

[Henrik Bengtsson]

On Tue, Dec 21, 2010 at 10:15 AM, Henrik Bengtsson <hb at biostat.ucsf.edu> wrote:
> Hi,
>
> I am using Octave; what does that save options do, more specifically,
> is compression taking place when saving that file?

That should be: I am [not] using Octave... /H

>
> If compression is done, then the Rcompression package is utilized by
> R.matlab (otherwise not).  BTW, you don't have to load Rcompression
> explicitly; R.matlab will do it for you if needed.  So, if you start a
> fresh R session and load R.matlab and then try to load your package,
> is Rcompression loaded?  If so, what version Rcompression do you have
> installed, i.e. what does
>
> sessionInfo()
>
> report afterward?   Duncan TL did Rcompression updates addressing
> memory usage about a year ago (I think) and it might be that you are
> using an older version of it.  You should also update R.matlab et al,
> because your using old versions (though I don't think that is the
> cause here).
>
> If Rcompression is the cause here, then it also make sense that you
> don't experience the memory hog when reading a text file (which is
> never compressed).  You could also see if there is an option in Octave
> that safes to binary format but without compression.  I know Matlab
> has such options.
>
> /Henrik
> (author of R.matlab)
>
>
>
>
> On Mon, Dec 20, 2010 at 7:11 AM, Stefano Ghirlanda
> <dr.ghirlanda at gmail.com> wrote:
>> Hi Ben,
>> Thanks for your reply. My data structure is about 20000 x 2000 so one
>> order of magnitude the one you tried. I have no problem saving and
>> reading smaller data structures (even large ones, just not his large)
>> between octave and R using octave's "save -7" (which saves MATLAB v5
>> files) and R.matlab's readMat. And I can save in text format in octave
>> and read in R using read.octave (from package foreign) so it's not a
>> big deal. I was just surprised that R.matlab needed more memory than I
>> have (I have 3GB on this machine).
>>
>> Thanks,
>> Stefano
>>
>> On Sun, Dec 19, 2010 at 10:54 PM, Ben Bolker <bbolker at gmail.com> wrote:
>>> Stefano Ghirlanda <dr.ghirlanda <at> gmail.com> writes:
>>>
>>>> I am trying to load into R a MATLAB format file (actually, as saved by
>>>> octave). The file is about 300kB but R complains with a memory
>>>> allocation error:
>>>>
>>>> > library(Rcompression)
>>>> > library(R.matlab)
>>>> Loading required package: R.oo
>>>> Loading required package: R.methodsS3
>>>> R.methodsS3 v1.2.0 (2010-03-13) successfully loaded. See ?R.methodsS3 for
>>>> help.
>>>> R.oo v1.7.2 (2010-04-13) successfully loaded. See ?R.oo for help.
>>>> R.matlab v1.3.1 (2010-04-20) successfully loaded. See ?R.matlab for help.
>>>> > f <- readMat("freq.mat")
>>>> Error: cannot allocate vector of size 296.5 Mb
>>>>
>>>> On the other hand, if I save the same data in ascii format (from
>>>> octave: "save -text"), resulting in a 75MB file, then I can load it
>>>> without problems with the read.octave() function from package foreign.
>>>> Is this a known issue or am I doing something wrong? My R version is:
>>>
>>>  This is not a package I'm particularly familiar with, but:
>>>
>>>  what commands did you use to save the file in octave?  Based on
>>> 'help save' I think that 'save' by default would get you an octave
>>> format file ... you might have to do some careful reading in
>>> ?readMat (in R) and 'help save' (in octave) to figure out the
>>> correspondence between octave/MATLAB and R/MATLAB.
>>>   If possible, try saving a small file and see if it works; if
>>> you still don't know what's going on, post that file somewhere for
>>> people to try.
>>>
>>>  I was able to
>>>
>>> save -6 "save.mat" in octave and
>>> readMat("save.mat") in R successfully,
>>> saving a vector of integers from 1 to 1 million (which
>>> took about 7.7 Mb)
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>>
>>
>> --
>> Stefano Ghirlanda
>> www.intercult.su.se/~stefano - drghirlanda.wordpress.com
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>