[R-pkg-devel] Logging data to disk from C++ using a DBMS

[Iñaki Úcar]

Hi all,

We have a simulator, 'simmer', implemented as a C++ object (with Rcpp).
This simulator generates a bunch of data, and there are some functions
defined to bring this data to R space as data frames. The memory usage can
be a problem depending on the simulation model, so we would like to add
support for logging data directly to disk while being as transparent as
possible from the user perspective.

Our idea is to define a reusable chunk size and periodically append the
chunk to a DBIConnection by calling DBI::dbWriteTable from C++. In this
way, the subsequent analysis may be performed with dplyr no matter which
storage method is selected, memory or database.

The question is whether there is a better, more efficient, way of achieving
this goal. Is there any package already doing something similar? Ideas and
alternatives welcome. Thanks in advance.

Regards,
-- 
Iñaki Úcar
http://www.enchufa2.es
@Enchufa2

	[[alternative HTML version deleted]]