[R-sig-hpc] RMVL
Vladimir Dergachev
vo|ody@ @end|ng |rom m|nd@pr|ng@com
Fri Jul 9 18:03:44 CEST 2021
Got a new package on CRAN - would greatly appreciate feedback:
https://cran.r-project.org/package=RMVL
This allows to memory map datasets into the R process. On a cluster this
saves memory on the nodes, and saves time loading data when using
parallel package.
There is a companion library for C-based codes:
https://github.com/volodya31415/libMVL
The MVL file format was designed around 64-bit data, so the address space
is large enough to accomodate all storage available today.
Looking forward to comments and suggestions,
Vladimir Dergachev
More information about the R-sig-hpc
mailing list