[R] Problem with mclapply in package parallel

Harold Petithomme harold.petithomme at meteo.fr
Fri Jul 21 16:18:11 CEST 2017


Dear fellow R users,

I am experiencing some problem using function mclapply from recommended R package parallel. This problem is a bit "deep in source code" (maybe r-devel list would be more appropriate ?).

Actually, I have noticed on a specific Linux HPC system that the various readChild() calls in mclapply did not end the children processes, as would be expected (ie compared to what happens on some other Linux machines that I am used to use). Trying to localize the problem, I have recompiled R (R-3.2.5, Intel compilers) after having the 'MC_DEBUG' macro activated in C file src/fork.c. It seems that global variable child_can_exit (declared as static int) is waited for change in function mc_exit but this never happens due to compiler's optimization configuration. A solution might be to declare variable child_can_exit as "static int volatile"; my tests work fine indeed.

This is what I have done locally (ie no system-wide installation of this patched R) for my use. Do some users have already noticed this problem ? What can be done for patching recent R versions, if appropriate ?

Supplemental info at R 'make' step and then run :
Machine : Linux [macname] 2.6.32-642.15.1.el6.x86_64 #1 SMP Mon Feb 20 02:26:38 EST 2017 x86_64 x86_64 x86_64 GNU/Linux
Compiler : Intel(R) C Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 16.0.1.150
Compiling : icc -std=gnu99 -I../../../../include -DNDEBUG -I../../../include -I../../../../src/include -DHAVE_CONFIG_H -I/usr/local/include    -fpic  -g -O2 -std=c99  -c fork.c -o fork.o
icc: command line warning #10121: overriding '-std=gnu99' with '-std=c99'
Linkin' : icc -std=gnu99 -shared -L/usr/local/lib64 -o parallel.so init.o rngstream.o fork.o
Running : mclapply's simple example under R-3.2.5

Thank you for advice or discussion.
Kind regards,
Harold Petithomme


********************************************************* 
Harold PETITHOMME 
Equipe Données de Prévision (DPREVI/COMPAS/DOP) 

Météo France - Direction de la Production 
42, avenue G. Coriolis. 
31057 Toulouse Cedex 
France 

Tel : (33/0)5.61.07.82.85 
Fax : (33/0)5.61.07.86.09 
E-mail : harold.petithomme at meteo.fr 
********************************************************* 



More information about the R-help mailing list