[Rd] Single-threaded aspect

[Charles Determan]

Thank you Simon for the detailed reply.  That explains much more of what I
was looking for from the R side.

Dirk, I'm sorry if I seem hung up on anything here but I am trying to
understand the details.  My reply about XPtr or XPtr on arma/Eigen was to
confirm my understanding was correct, which it appears it was.  I was not
aware the RVector/RMatrix objects don't connect to R as I am just now
familiarizing myself with the package, that explains more of my confusion.
I will look at doing work within the compiled code as you have suggested.

Regards,
Charles

On Thu, May 12, 2016 at 9:18 AM, Dirk Eddelbuettel <edd at debian.org> wrote:

>
> On 12 May 2016 at 13:11, Mark van der Loo wrote:
> | Charles,
> |
> | 1. Perhaps this question is better directed at the R-help or
> | R-pacakge-devel mailinglist.
> |
> | 2. It basically means that R itself can only evaluate one R expression at
> | the time.
> |
> | The parallel package circumvents this by starting multiple R-sessions and
> | dividing workload.
> |
> | Compiled code called by R (such as C++ code through RCpp or C-code
> through
> | base R's interface) can execute multi-threaded code for internal
> purposes,
> | using e.g. openMP. A limitation is that compiled code cannot call R's C
> API
> | from multiple threads (in many cases). For example, it is not thread-safe
> | to create R-variables from multiple threads running in C. (R's variable
> | administration is such that the order of (un)making them from compiled
> code
> | matters).
>
> Well put.
>
> | I am not very savvy on Rcpp or XPtr objects, but it appears that Dirk
> | provided answers about that in your SO-question.
>
> Charles seems to hang himself up completely about a small detail, failing
> to
> see the forest for the trees.
>
> There are (many) working examples of parallel (compiled) code with R. All
> of
> them stress (and I simplify here) that can you touch R objects, or call
> back
> into R, for fear of any assignment or allocation triggering an R event.  R
> being single-threaded it cannot do this.
>
> My answer to this problem is to only use non-R data structures. That is
> what
> RcpParallel does in the actual parallel code portions in all examples --
> types RVector and RMatrix do NOT connect back to R. There are several
> working
> examples.  That is also what the OpenMP examples at the Rcpp Gallery do.
>
> Charles seems to be replying 'but I use XPtr' or 'I use XPtr on arma::mat
> or
> Eigen::Matrixxd' and seems to forget that these are proxy objects to SEXPs.
> XPtr just wrap the SEXP for external pointers; Arma's and Eigen's matrices
> are performant via RcppArmadillo and RcppEigen because we use R memory via
> proxies.  All of that is 'too close to R' for comfort.
>
> So the short answer is:  enter compiled code from R, set a mutex (either
> conceptually or explicitly), _copy_ your data in to plain C++ data
> structures
> and go to town in parallel via OpenMP and other multithreaded approaches.
> Then collect the result, release the mutex and move back up.
>
> I hope this help.
>
> Dirk
>
> |
> | Best,
> | Mark
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> | Op do 12 mei 2016 om 14:46 schreef Charles Determan <
> cdetermanjr at gmail.com>:
> |
> | > R Developers,
> | >
> | > Could someone help explain what it means that R is single threaded?  I
> am
> | > trying to understand what is actually going on inside R when users
> want to
> | > parallelize code.  For example, using mclapply or foreach (with some
> | > backend) somehow allows users to benefit from multiple CPUs.
> | >
> | > Similarly there is the RcppParallel package for RMatrix/RVector
> objects.
> | > But none of these address the general XPtr objects in Rcpp.  Some
> readers
> | > here may recognize my question on SO (
> | >
> | >
> http://stackoverflow.com/questions/37167479/rcpp-parallelize-functions-that-return-xptr
> | > )
> | > where I was curious about parallel calls to C++/Rcpp functions that
> return
> | > XPtr objects.  I am being a little more persistent here as this
> limitation
> | > provides a very hard stop on the development on one of my packages that
> | > heavily uses XPtr objects.  It's not meant to be a criticism or
> intended to
> | > be rude, I just want to fully understand.
> | >
> | > I am willing to accept that it may be impossible currently but I want
> to at
> | > least understand why it is impossible so I can explain to future users
> why
> | > parallel functionality is not available.  Which just echos my original
> | > question, what does it mean that R is single threaded?
> | >
> | > Kind Regards,
> | > Charles
> | >
> | >         [[alternative HTML version deleted]]
> | >
> | > ______________________________________________
> | > R-devel at r-project.org mailing list
> | > https://stat.ethz.ch/mailman/listinfo/r-devel
> | >
> |
> |       [[alternative HTML version deleted]]
> |
> | ______________________________________________
> | R-devel at r-project.org mailing list
> | https://stat.ethz.ch/mailman/listinfo/r-devel
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
>

	[[alternative HTML version deleted]]