[R-sig-hpc] parallel and openblas

[Stephen Weston]

There's an interesting discussion entitled "all processes run on
one CPU core" at:

    https://github.com/ipython/ipython/issues/840

Someone was experiencing a very similar problem to the one that
Claudia described using GotoBLAS2 with IPython and NumPy.
Apparently it was fixed by recompiling GotoBLAS2 with the
"NO_AFFINITY" parameter set to "1" in Makefile.rule, and then
rebuilding "NumPy".

It seems pretty strange, but GotoBLAS2/OpenBLAS may be modifying
the affinity of the R process by calling sched_setaffinity() when
it is initialized, and that is causing the problems that Claudia
and Martin have seen.

So perhaps the solution is to recompile GotoBLAS2/OpenBLAS with
NO_AFFINITY=1, and then rebuild R with it.

- Steve


On Tue, Apr 24, 2012 at 12:00 PM, beleites,claudia
<claudia.beleites at ipht-jena.de> wrote:
> Martin,
>
> please let me know the permanent solution - I'm still using taskset all the time (I don't have admin rights, but our admin would do it if I could tell him what exactly I need.).
>
> Claudia
> ________________________________________
> Von: Martin Renner [greatauklet at gmail.com]
> Gesendet: Dienstag, 24. April 2012 17:40
> An: beleites,claudia
> Cc: r-sig-hpc at r-project.org
> Betreff: Re: [R-sig-hpc] parallel and openblas
>
> Hallo Claudia,
>
> Thank you for that hint -- this works! A bit more permanent solution would be nice though; will need to look into that.
>
> Best,
> Martin
>
>
>
> Martin Renner
> Post-doctoral Fellow                            phone: 907-226 4672
> University of Washington                           or: 907-235 0728
> School of Aquatic and Fishery Sciences          Seattle, USA
>
>
> On 24 Apr 2012, at 02:29 , beleites,claudia wrote:
>
>> Martin,
>>
>> I possibly have/had the same problem (on a CentOS 5 system).
>> The R was assigned to one core only, and the solution is:
>> system(sprintf('taskset -p 0xffffffff %d', Sys.getpid()))
>>
>> The whole thread is this: https://stat.ethz.ch/pipermail/r-sig-hpc/2011-November/001171.html
>>
>> Claudia
>>
>>
>> ________________________________________
>> Von: Martin Renner [greatauklet at gmail.com]
>> Gesendet: Dienstag, 24. April 2012 10:51
>> An: beleites,claudia
>> Cc: r-sig-hpc at r-project.org
>> Betreff: Re: [R-sig-hpc] parallel and openblas
>>
>> Yes, Claudia, you're probaply right that all these threads are on one core. Yes, %*% is working as expected, utilizing all available cores, when R is linked to openblas.
>>
>> Martin
>>
>>
>>
>> On 23 Apr 2012, at 12:07 , Claudia Beleites wrote:
>>
>>> Martin, do you actually know each core works with 1/8th (as opposed to
>>> all 8 threads are run on the same core)?
>>>
>>> Is implicitly parallel stuff (e.g. %*%) without package parallel working OK?
>>>
>>> Claudia
>>>
>>> Am 23.04.2012 21:53, schrieb Martin Renner:
>>>> Parallel and openblas don't seem to mix well on my machine. If I link openblas, a job executed through parallel (using either the multicore or snow (local socket cluster) setup), each of my 8 cores only operates at 1/8 of 100% (taking a little longer than serial execution). Linking to the reference blas or to single-threaded atlas does not cause this handicap when running snow or multicore.
>>>>
>>>> Is this a known problem (My google attempts were fruitless)? If yes, is there a fix for it? Do MKL or multi-threaded atlas have the same issues?
>>>>
>>>> Thank you for your time.
>>>>
>>>> Martin
>>>>
>>>>
>>>>
>>>> Martin Renner
>>>> Post-doctoral Fellow                         phone: 907-226 4672
>>>> University of Washington                        or: 907-235 0728
>>>> School of Aquatic and Fishery Sciences               Seattle, USA
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> debian squeeze on 8-core Xeon
>>>>> sessionInfo()
>>>> R version 2.15.0 (2012-03-30)
>>>> Platform: x86_64-unknown-linux-gnu (64-bit)
>>>>
>>>> locale:
>>>> [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>>>> [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
>>>> [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
>>>> [7] LC_PAPER=C                 LC_NAME=C
>>>> [9] LC_ADDRESS=C               LC_TELEPHONE=C
>>>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>>>
>>>> attached base packages:
>>>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>>> _______________________________________________
>>>> R-sig-hpc mailing list
>>>> R-sig-hpc at r-project.org
>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>>>
>>>
>>> --
>>> Claudia Beleites
>>> Spectroscopy/Imaging
>>> Institute of Photonic Technology
>>> Albert-Einstein-Str. 9
>>> 07745 Jena
>>> Germany
>>>
>>> email: claudia.beleites at ipht-jena.de
>>> phone: +49 3641 206-133
>>> fax:   +49 2641 206-399
>>>
>>>
>>>
>>>
>>
>
> _______________________________________________
> R-sig-hpc mailing list
> R-sig-hpc at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc