[R-pkg-devel] Logging data to disk from C++ using a DBMS

Iñaki Úcar i.ucar86 at gmail.com
Sat Jun 3 17:45:06 CEST 2017

Hi all,

We have a simulator, 'simmer', implemented as a C++ object (with Rcpp).
This simulator generates a bunch of data, and there are some functions
defined to bring this data to R space as data frames. The memory usage can
be a problem depending on the simulation model, so we would like to add
support for logging data directly to disk while being as transparent as
possible from the user perspective.

Our idea is to define a reusable chunk size and periodically append the
chunk to a DBIConnection by calling DBI::dbWriteTable from C++. In this
way, the subsequent analysis may be performed with dplyr no matter which
storage method is selected, memory or database.

The question is whether there is a better, more efficient, way of achieving
this goal. Is there any package already doing something similar? Ideas and
alternatives welcome. Thanks in advance.

Iñaki Úcar

	[[alternative HTML version deleted]]

More information about the R-package-devel mailing list