[Rd] Single-threaded aspect

[Dirk Eddelbuettel]

On 12 May 2016 at 09:18, Dirk Eddelbuettel wrote:
| 
| On 12 May 2016 at 13:11, Mark van der Loo wrote:
| | Charles,
| | 
| | 1. Perhaps this question is better directed at the R-help or
| | R-pacakge-devel mailinglist.
| | 
| | 2. It basically means that R itself can only evaluate one R expression at
| | the time.
| | 
| | The parallel package circumvents this by starting multiple R-sessions and
| | dividing workload.
| | 
| | Compiled code called by R (such as C++ code through RCpp or C-code through
| | base R's interface) can execute multi-threaded code for internal purposes,
| | using e.g. openMP. A limitation is that compiled code cannot call R's C API
| | from multiple threads (in many cases). For example, it is not thread-safe
| | to create R-variables from multiple threads running in C. (R's variable
| | administration is such that the order of (un)making them from compiled code
| | matters).
| 
| Well put.
| 
| | I am not very savvy on Rcpp or XPtr objects, but it appears that Dirk
| | provided answers about that in your SO-question.
| 
| Charles seems to hang himself up completely about a small detail, failing to
| see the forest for the trees.
| 
| There are (many) working examples of parallel (compiled) code with R. All of
| them stress (and I simplify here) that can you touch R objects, or call back

An import 'not' missing here (and a reordering);  "that you CANNOT touch R objects"

Sorry, Dirk

| into R, for fear of any assignment or allocation triggering an R event.  R
| being single-threaded it cannot do this.
| 
| My answer to this problem is to only use non-R data structures. That is what
| RcpParallel does in the actual parallel code portions in all examples --
| types RVector and RMatrix do NOT connect back to R. There are several working
| examples.  That is also what the OpenMP examples at the Rcpp Gallery do.
| 
| Charles seems to be replying 'but I use XPtr' or 'I use XPtr on arma::mat or
| Eigen::Matrixxd' and seems to forget that these are proxy objects to SEXPs.
| XPtr just wrap the SEXP for external pointers; Arma's and Eigen's matrices
| are performant via RcppArmadillo and RcppEigen because we use R memory via
| proxies.  All of that is 'too close to R' for comfort.
| 
| So the short answer is:  enter compiled code from R, set a mutex (either
| conceptually or explicitly), _copy_ your data in to plain C++ data structures
| and go to town in parallel via OpenMP and other multithreaded approaches.
| Then collect the result, release the mutex and move back up.
| 
| I hope this help.
| 
| Dirk
| 
| | 
| | Best,
| | Mark
| | 
| | 
| | 
| | 
| | 
| | 
| | 
| | 
| | 
| | 
| | Op do 12 mei 2016 om 14:46 schreef Charles Determan <cdetermanjr at gmail.com>:
| | 
| | > R Developers,
| | >
| | > Could someone help explain what it means that R is single threaded?  I am
| | > trying to understand what is actually going on inside R when users want to
| | > parallelize code.  For example, using mclapply or foreach (with some
| | > backend) somehow allows users to benefit from multiple CPUs.
| | >
| | > Similarly there is the RcppParallel package for RMatrix/RVector objects.
| | > But none of these address the general XPtr objects in Rcpp.  Some readers
| | > here may recognize my question on SO (
| | >
| | > http://stackoverflow.com/questions/37167479/rcpp-parallelize-functions-that-return-xptr
| | > )
| | > where I was curious about parallel calls to C++/Rcpp functions that return
| | > XPtr objects.  I am being a little more persistent here as this limitation
| | > provides a very hard stop on the development on one of my packages that
| | > heavily uses XPtr objects.  It's not meant to be a criticism or intended to
| | > be rude, I just want to fully understand.
| | >
| | > I am willing to accept that it may be impossible currently but I want to at
| | > least understand why it is impossible so I can explain to future users why
| | > parallel functionality is not available.  Which just echos my original
| | > question, what does it mean that R is single threaded?
| | >
| | > Kind Regards,
| | > Charles
| | >
| | >         [[alternative HTML version deleted]]
| | >
| | > ______________________________________________
| | > R-devel at r-project.org mailing list
| | > https://stat.ethz.ch/mailman/listinfo/r-devel
| | >
| | 
| | 	[[alternative HTML version deleted]]
| | 
| | ______________________________________________
| | R-devel at r-project.org mailing list
| | https://stat.ethz.ch/mailman/listinfo/r-devel
| 
| -- 
| http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
| 
| ______________________________________________
| R-devel at r-project.org mailing list
| https://stat.ethz.ch/mailman/listinfo/r-devel

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org