[R] Avoiding a memory copy by [[

[Prof Brian Ripley]

On Tue, 23 May 2006, Henrik Bengtsson wrote:

> On 5/23/06, Matthew Dowle <mdowle at concordiafunds.com> wrote:
>> 
>> Thanks.
>> 
>> I looked some more and found that L$b[1:10] doesn't seem to copy L$b.  If
>> that's correct why does L[[2]][1:10] copy L[[2]] ?
>
> I forgot, this is probably what I was told in discussion about
> UseMethod("$") the other day: The "$" operator is very special. Its
> second argument (the one after the operator) is not evaluated.  For
> "[[" it is.  This is probably also why the solution with environment
> works.  I think some with the more knowledge about the R core has to
> give you the details on this, and especially why "$" is special in the
> first place (maybe because of the example you're giving).

That's not the reason here: the internal code for [[ duplicates for vector 
lists but not pairlists.  That could be replaced by a NAMED optimization, 
although we would not do that until 2.4.0 (for which Thomas Lumley has 
written profiling code for memory use and duplication).

>
> /Henrik
>
>> > -----Original Message-----
>> > From: Prof Brian Ripley [mailto:ripley at stats.ox.ac.uk]
>> > Sent: 23 May 2006 16:23
>> > To: Matthew Dowle
>> > Cc: 'r-help at stat.math.ethz.ch'
>> > Subject: Re: [R] Avoiding a memory copy by [[
>> >
>> >
>> > On Tue, 23 May 2006, Matthew Dowle wrote:
>> >
>> > >
>> > > Hi,
>> > >
>> > > n = 10000000
>> > > L = list(a=integer(n), b=integer(n))
>> > >
>> > > L[[2]][1:10]  gives me the first 10 items of the 2nd vector in the
>> > > list L. It works fine.  However it appears to copy the
>> > entire L[[2]]
>> > > vector in memory first, before subsetting it.  It seems reasonable
>> > > that "[[" can't know that all that is to be done is to do [1:10] on
>> > > the result and therefore a copy in memory of the entire
>> > vector L[[2]]
>> > > is not required.  Only a new vector length 10 need be
>> > created.  I see
>> > > why [[ needs to make a copy in general.
>> > >
>> > > L[[c(2,1)]]  gives me the 1st item of the 2nd vector in the
>> > list L.
>> > > It works fine,  and does not appear to copy L[[2]] in
>> > memory first.
>> > > Its much faster as n grows large.
>> > >
>> > > But I need more than 1 element of the vector ....
>> > L[[c(2,1:10)]]   fails
>> > > with "Error: recursive indexing failed at level 2"
>> >
>> > Note that [[ ]] is documented to only ever return one
>> > element, so this is
>> > invalid.
>> >
>> > > Is there a way I can obtain the first 10 items of L[[2]] without a
>> > > memory copy of L[[2]]  ?
>> >
>> > Use .Call
>> >
>> > --
>> > Brian D. Ripley,                  ripley at stats.ox.ac.uk
>> > Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
>> > University of Oxford,             Tel:  +44 1865 272861 (self)
>> > 1 South Parks Road,                     +44 1865 272866 (PA)
>> > Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>> >
>> 
>> ______________________________________________
>> R-help at stat.math.ethz.ch mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide! 
>> http://www.R-project.org/posting-guide.html
>> 
>> 
>
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595