[R-sig-hpc] multicore package and C/C++ code: any special care?

[Renaud Gaujoux]

Ok, thanks. So it should be all fine.
I do make use of shared memory objects (with the package bigmemory) but 
it only on the R side in a %dopar% loop (doMC).
The C++ code is independent of it and only used in part of the 
inner-loop computation, I deal with concurrent access to these shared 
objects via rw.mutex objects (also from bigmemory):

# basically this gets the best (max) result among 100
compute.all <- function(x, ...){

                ## 1. SETUP                              
                # - Define a shared memory objects
                best.shared <- shared.big.matrix(1, 1, type='double', 
init=NA)           
                   
                # - Define a mutex to control the access to the shared 
memory objects
                mut <- rw.mutex()
                mut.desc <- bigmemory::describe(mut)           
                   
                ## 2. RUN
                res.runs <- foreach(n=1:100) %dopar% {
                   
                    # here is the C++ part
                    res <- cppcomputation(x, ...)
                   
                   # load shared object
                   best.shared <- attach.big.matrix(best.desc)
                       
                   ##LOCK_MUTEX                   
                   # retrieve and lock the mutex
                   mut <- attach.rw.mutex(mut.desc)   
                   rwlock(mut)
                       
                   # check if the result is greater than the previous max
                   best <- best.shared[]
                   if( is.na(best) || res > best ){
                      
                       # update residuals
                       best.shared[] <- res
                                                      
                   }
                   else res <- NULL
                       
                   
                   # unlock the mutex
                   bigmemory::unlock(mut)
                   ##END_LOCK_MUTEX
                                   
                   # return the result
                  res
            }

             res.runs
}           


Bests,
Renaud

Brian G. Peterson wrote:
> Renaud Gaujoux wrote:
>> Hi,
>>
>> my package aims to be used in a multicore and/or computer-cluster 
>> environment.
>> I'm re-implementing part of the code in C++. Apparently it's running 
>> ok in multicore mode, but I want to be sure there is no potential 
>> problem with this, on any platform.
>> Is there any special things to worry about (thread-safe, memory, etc, 
>> ...)?
>> Thanks.
>>
>> Bests,
>> Renaud
>>
>> PS: even if there is nothing to worry about, I'd be happy to get some 
>> insights on the reason behind it.
> 'multicore' spawns separate R processes to run for each core.  Your 
> custom C/C++ code should not make use of shared memory blocks that 
> could be overwritten by one of the other R processes, or expect 
> interprocess communication to work between the processes without 
> explicit handling.  If all your memory is explicitly allocated, used, 
> and cleaned up, you should have no problems.  Loop dependence, or 
> dependence on some shared global in the calling R environment, should 
> be avoided.  Be explicit about both inputs and outputs to your code.
>
> Others are definitely more qualified to get at the nuances, but this 
> should cover the broad strokes.
>
> Regards,
>
>    - Brian
>