[R-sig-hpc] RMVL

Vladimir Dergachev vo|ody@ @end|ng |rom m|nd@pr|ng@com
Fri Jul 9 18:03:44 CEST 2021


Got a new package on CRAN - would greatly appreciate feedback:

https://cran.r-project.org/package=RMVL

This allows to memory map datasets into the R process. On a cluster this 
saves memory on the nodes, and saves time loading data when using 
parallel package.

There is a companion library for C-based codes:

https://github.com/volodya31415/libMVL

The MVL file format was designed around 64-bit data, so the address space 
is large enough to accomodate all storage available today.

Looking forward to comments and suggestions,

Vladimir Dergachev



More information about the R-sig-hpc mailing list