[R] save an object by dynamicly created name
Henrik Bengtsson
hb at stat.berkeley.edu
Sat Dec 12 22:31:37 CET 2009
saveObject() and loadObject() are using save() and load() in base.
The default is to compress the data when saving, which take some time.
Using saveObject(..., compress=FALSE) is probably faster, but takes
up more disk space. ...and make sure you don't work towards a file
system over a network, because that can slow things down (doesn't
sound like you do).
/H
On Sun, Nov 8, 2009 at 6:26 AM, Hao Cen <hcen at andrew.cmu.edu> wrote:
> Hi Henrik,
>
> I am using your saveObject/loadObject to handle over 1000 matrices. It
> worked beautifully. Because I need to load those matrices often for
> evaluating a few functions on them and those matrices do not fit all in
> memory at once, is there a way to speed up the loading part? I tried save
> all the binary files to /dev/shm (shared memory section in linux) but the
> speed of loadObject on /dev/shm remains the same as on the disk.
>
> Thanks
>
> Hao
>
>
>
> -----Original Message-----
> From: henrik.bengtsson at gmail.com [mailto:henrik.bengtsson at gmail.com] On
> Behalf Of Henrik Bengtsson
> Sent: Monday, November 02, 2009 12:34 AM
> To: David Winsemius
> Cc: r-help at r-project.org; jeffc
> Subject: Re: [R] save an object by dynamicly created name
>
> On Sun, Nov 1, 2009 at 9:18 PM, David Winsemius <dwinsemius at comcast.net>
> wrote:
>>
>> On Nov 1, 2009, at 11:28 PM, Henrik Bengtsson wrote:
>>
>>> On Sun, Nov 1, 2009 at 7:48 PM, David Winsemius <dwinsemius at comcast.net>
>>> wrote:
>>>>
>>>> On Nov 1, 2009, at 10:16 PM, Henrik Bengtsson wrote:
>>>>
>>>>> path <- "data";
>>>>> dir.create(path);
>>>>>
>>>>> for (i in 1:10) {
>>>>> m <- i:5;
>>>>> filename <- sprintf("m%02d.Rbin", i);
>>>>> pathname <- file.path(path, filename);
>>>>> save(m, file=pathname);
>>>>> }
>>>>>
>>>>
>>>> That would result in each of the ten files containing an object with the
>>>> same name == "m". (Also on my system R data files have type Rdta.) So I
>>>> thought what was requested might have been a slight mod:
>>>>
>>>> path <- "~/";
>>>> dir.create(path);
>>>>
>>>> for (i in 1:10) {
>>>> assign( paste("m", i, sep=""), i:5)
>>>> filename <- sprintf("m%02d.Rdta", i)
>>>> pathname <- file.path(path, filename)
>>>> obj =get(paste("m", i, sep=""))
>>>> save(obj, file=pathname)
>>>> }
>>>
>>> Then a more convenient solution is to use saveObject() and
>>> loadObject() of R.utils. saveObject() does not save the name of the
>>> object save.
>>
>> The OP asked for this outcome :
>>
>> " I would like to save m as m1, m2, m3 ...,
>> to file /home/data/m1, /home/data/m2, home/data/m3, ..."
>>
>>
>>> If you want to save multiple objects, the wrap them up
>>> in a list.
>>
>> I agree that a list would makes sense if it were to be stored in one file
> ,
>> although it was not what requested.
>
> That comment was not for the OP, but for saveObject()/loadObject() in
> general.
>
>> But wouldn't that require assign()-ing a name before list()-wrapping?
>
> Nope, the whole point of using saveObject()/loadObject() is to save
> the objects/values without their names that you happens to choose in
> the current session, and to avoid overwriting existing ones in your
> next session. My example could also have been:
>
> library("R.utils");
> saveObject(list(a=1,b=LETTERS,c=Sys.time()), file="foo.Rbin");
> y <- loadObject("foo.Rbin");
> z <- loadObject("foo.Rbin");
> stopifnot(identical(y,z));
>
> If you really want to attach the elements of the saved list, do:
>
> attachLocally(loadObject("foo.Rbin"));
>> str(a)
> num 1
>> str(b)
> chr [1:26] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" ...
>> str(c)
> POSIXct[1:1], format: "2009-11-01 21:30:41"
>
>>
>> I suppose we ought to mention that the use of assign to create a variable
> is
>> a FAQ ... 7.21? Yep, I have now referred to it a sufficient number of
> times
>> to refer to it by number.
>>
>>
> http://cran.r-project.org/doc/FAQ/R-FAQ.html#How-can-I-turn-a-string-into-a-
> variable_003f
>
> My personal take on assign() and get() is that if you find yourself
> using them (at this level), there is a good chance there exists a
> better solution that you should use instead.
>
> My $.02
>
> /H
>
>>
>> --
>> David
>>
>>> loadObject() does not assign variable, but instead return
>>> them. Example:
>>>
>>> library("R.utils");
>>> x <- list(a=1,b=LETTERS,c=Sys.time());
>>> saveObject(x, file="foo.Rbin");
>>> y <- loadObject("foo.Rbin");
>>> stopifnot(identical(x,y));
>>
>>>
>>> So, for the original example, I'd recommend:
>>>
>>> library("R.utils");
>>> path <- "data";
>>> mkdirs(path);
>>>
>>> for (i in 1:10) {
>>> m <- i:5;
>>> filename <- sprintf("m%02d.Rbin", i);
>>> saveObject(m, file=filename, path=path);
>>> }
>>>
>>> and loading the objects back as:
>>>
>>> for (i in 1:10) {
>>> filename <- sprintf("m%02d.Rbin", i);
>>> m <- loadObject(filename, path=path);
>>> print(m);
>>> }
>>> /Henrik
>>>
>>>>
>>>> --
>>>> David.
>>>>
>>>>> /H
>>>>>
>>>>> On Sun, Nov 1, 2009 at 6:53 PM, jeffc <hcen at andrew.cmu.edu> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I would like to save a few dynamically created objects to disk. The
>>>>>> following is the basic flow of the code segment
>>>>>>
>>>>>> for(i = 1:10) {
>>>>>> m = i:5
>>>>>> save(m, file = ...) ## ???
>>>>>> }
>>>>>> To distinguish different objects to be saved, I would like to save m
> as
>>>>>> m1,
>>>>>> m2, m3 ..., to file /home/data/m1, /home/data/m2, home/data/m3, ...
>>>>>>
>>>>>> I tried a couple of methods on translating between object names and
>>>>>> strings
>>>>>> (below) but couldn't get it to work.
>>>>>> https://stat.ethz.ch/pipermail/r-help/2008-November/178965.html
>>>>>> http://tolstoy.newcastle.edu.au/R/help/04/08/2673.html
>>>>>>
>>>>>> Any suggestions would be appreciated.
>>>>>>
>>>>>> thanks
>>>>>>
>>>>>> Hao
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>>
>>>>>>
> http://old.nabble.com/save-an-object-by-dynamicly-created-name-tp26155437p26
> 155437.html
>>>>>> Sent from the R help mailing list archive at Nabble.com.
>>>>>>
>>>>>> ______________________________________________
>>>>>> R-help at r-project.org mailing list
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>>>> PLEASE do read the posting guide
>>>>>> http://www.R-project.org/posting-guide.html
>>>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>>>
>>>>>
>>>>> ______________________________________________
>>>>> R-help at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>>> PLEASE do read the posting guide
>>>>> http://www.R-project.org/posting-guide.html
>>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>
>>>> David Winsemius, MD
>>>> Heritage Laboratories
>>>> West Hartford, CT
>>>>
>>>> ______________________________________________
>>>> R-help at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide
>>>> http://www.R-project.org/posting-guide.html
>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>
>>
>> David Winsemius, MD
>> Heritage Laboratories
>> West Hartford, CT
>>
>>
>
>
>
More information about the R-help
mailing list