[Rd] [External] Any plans for ALTREP lists (VECSXP)?

[Tierney, Luke]

Eventually, but probably not in the next release. There are many more
issues to think through for vectors where the elements can be
arbitrary R object, and I don't think there will be time for that soon
given other issues on the table.

Best,

luke

On Tue, 23 Jul 2019, Bemis, Kylie wrote:

> Hello,
>
> I was wondering if there were any plans for ALTREP lists (VECSXP)?
>
> It seems to me that they could be supported in a similar way to how ALTSTRING works, with Elt() and Set_elt() methods, or would there be some problems with that I’m not seeing due to lists not being atomic vectors?
>
> I was taking an approach of converting each list element (of a file-based list data structure) to an ALTREP representation to build up an “ALTREP list”.
>
> This seems fine for shorter lists with large elements, but I noticed that for longer lists with smaller elements, this could be far more time-consuming than simply reading the entire list into memory and returning a non-ALTREP list:
>
>> x
> <34840 length> matter_list :: out-of-memory list
> (1.1 MB real | 543.3 MB virtual)
>
>> system.time(y <- as.list(x))
>   user  system elapsed
>  1.116   2.175   5.053
>
>> system.time(z <- as.altrep(x))
>   user  system elapsed
> 36.295   4.717  41.216
>
>> .Internal(inspect(y))
> @108255000 19 VECSXP g1c7 [MARK,NAM(7)] (len=34840, tl=0)
>  @7f9044d9fc00 14 REALSXP g1c7 [MARK] (len=1129, tl=0) 404.093,404.096,404.099,404.102,404.105,...
>  @7f9044d25e00 14 REALSXP g1c7 [MARK] (len=890, tl=0) 409.924,409.927,409.931,409.934,409.937,...
>  @7f9044da6000 14 REALSXP g1c7 [MARK] (len=1878, tl=0) 400.3,400.303,400.306,400.309,400.312,...
>  @7f9031a6b000 14 REALSXP g1c7 [MARK] (len=2266, tl=0) 402.179,402.182,402.185,402.188,402.191,...
>  @7f9031a77a00 14 REALSXP g1c7 [MARK] (len=1981, tl=0) 403.021,403.024,403.027,403.03,403.033,...
>  ...
>
>> .Internal(inspect(z))
> @108210000 19 VECSXP g1c7 [MARK,NAM(7)] (len=34840, tl=0)
>  @7f904eea7660 14 REALSXP g1c0 [MARK,NAM(7)] matter vector (mode=4, len=1129, mem=0)
>  @7f9050347498 14 REALSXP g1c0 [MARK,NAM(7)] matter vector (mode=4, len=890, mem=0)
>  @7f904d286b20 14 REALSXP g1c0 [MARK,NAM(7)] matter vector (mode=4, len=1878, mem=0)
>  @7f904fd38820 14 REALSXP g1c0 [MARK,NAM(7)] matter vector (mode=4, len=2266, mem=0)
>  @7f904c75ce90 14 REALSXP g1c0 [MARK,NAM(7)] matter vector (mode=4, len=1981, mem=0)
>  ...
>
> In this situation, it would be much faster and simpler for me to return a theoretical ALTREP list that serves SEXP elements on-demand, similar to how ALTSTRING seems to be implemented.
>
> I don’t know how many other people would get a use out of ALTREP lists, but I certainly would.
>
> Are there any plans for this?
>
> Thanks!
>
> ~~~
> Kylie Ariel Bemis
> Khoury College of Computer Sciences
> Northeastern University
> kuwisdelu.github.io<https://kuwisdelu.github.io>
>
>
>
>
>
>
>
>
>
>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel using r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney using uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu