[R-sig-hpc] Rsge combined with mclapply?
Daniel Sabanés Bové
daniel.sabanesbove at ifspm.uzh.ch
Thu Jun 14 10:42:39 CEST 2012
thank you for this hint! I was not aware of this possibility. Yes,
indeed, then it should be trivial to let multiple jobs run on cores of
the same node. In the meantime I have found a solution by tweaking the
Rsge package to use mclapply() from the base R-package "parallel". This
-------- Original-Nachricht --------
Betreff: Re: [R-sig-hpc] Rsge combined with mclapply?
Von: Kasper Daniel Hansen <kasperdanielhansen at gmail.com>
An: Daniel Sabanés Bové <daniel.sabanesbove at ifspm.uzh.ch>
Datum: Mit 13 Jun 2012 23:10:23 CEST
> I am not too familiar with Rsge. But on our SGE cluster, the sysadmin
> has a specific type of parallel environment where you can do
> qsub -pe local 8-12
> which asks for 8-12 cores on the same node. My guess is that once
> such a parallel resource has been set up, it should be trivial to use
> with Rsge.
> On Tue, Jun 5, 2012 at 6:19 AM, Daniel Sabanés Bové
> <daniel.sabanesbove at ifspm.uzh.ch> wrote:
>> Dear Dan Bode,
>> first of all I would like to thank you very much for your R-package "Rsge".
>> It looks very helpful, because I need to submit my parallel jobs on the
>> local cluster using SGE.
>> Actually I have a trivially parallel problem which could be sequentially
>> solved by lapply(), and on a single machine with multiple cores by
>> mclapply() from the base R-package "parallel". Because the local cluster has
>> nodes with multiple cores, I would like to combine sge.parLapply (to
>> distribute to nodes) with mclapply (not lapply, to distribute to cores in
>> each node).
>> Before doing some modifications to sge.parParApply like introducing another
>> apply.method for mclapply, I would like to be sure that you have not already
>> programmed a nice solution for this problem. So, did you already have the
>> same problem as I am facing now? Would you think another apply.method for
>> sge.parParApply would be a good way to go?
>> Thank you very much in advance for any hints or suggestions.
>> Best regards
>> Daniel Sabanes
>> Daniel Sabanés Bové
>> University of Zurich
>> Institute of Social and Preventive Medicine
>> Division of Biostatistics
>> Hirschengraben 84
>> CH-8001 Zurich
>> Tel. +41 (0)44 634 49 73
>> Fax: +41 (0)44 634 43 86
>> URL : http://www.biostat.uzh.ch/aboutus/people/sabanes.html
>> R-sig-hpc mailing list
>> R-sig-hpc at r-project.org
More information about the R-sig-hpc