[R-sig-hpc] Rsge combined with mclapply?

[Daniel Sabanés Bové]

Hi Kasper,

thank you for this hint! I was not aware of this possibility. Yes, 
indeed, then it should be trivial to let multiple jobs run on cores of 
the same node. In the meantime I have found a solution by tweaking the 
Rsge package to use mclapply() from the base R-package "parallel". This 
also works.

Best regards
Daniel


-------- Original-Nachricht --------
Betreff: Re: [R-sig-hpc] Rsge combined with mclapply?
Von: Kasper Daniel Hansen <kasperdanielhansen at gmail.com>
An: Daniel Sabanés Bové <daniel.sabanesbove at ifspm.uzh.ch>
Datum: Mit 13 Jun 2012 23:10:23 CEST

> I am not too familiar with Rsge.  But on our SGE cluster, the sysadmin
> has a specific type of parallel environment where you can do
>    qsub -pe local 8-12
> which asks for 8-12 cores on the same node.  My guess is that once
> such a parallel resource has been set up, it should be trivial to use
> with Rsge.
>
> Kasper
>
> On Tue, Jun 5, 2012 at 6:19 AM, Daniel Sabanés Bové
> <daniel.sabanesbove at ifspm.uzh.ch>  wrote:
>> Dear Dan Bode,
>>
>> first of all I would like to thank you very much for your R-package "Rsge".
>> It looks very helpful, because I need to submit my parallel jobs on the
>> local cluster using SGE.
>>
>> Actually I have a trivially parallel problem which could be sequentially
>> solved by lapply(), and on a single machine with multiple cores by
>> mclapply() from the base R-package "parallel". Because the local cluster has
>> nodes with multiple cores, I would like to combine sge.parLapply (to
>> distribute to nodes) with mclapply (not lapply, to distribute to cores in
>> each node).
>>
>> Before doing some modifications to sge.parParApply like introducing another
>> apply.method for mclapply, I would like to be sure that you have not already
>> programmed a nice solution for this problem. So, did you already have the
>> same problem as I am facing now? Would you think another apply.method for
>> sge.parParApply would be a good way to go?
>>
>> Thank you very much in advance for any hints or suggestions.
>>
>> Best regards
>> Daniel Sabanes
>>
>> --
>> Daniel Sabanés Bové
>> University of Zurich
>> Institute of Social and Preventive Medicine
>> Division of Biostatistics
>> Hirschengraben 84
>> CH-8001 Zurich
>>
>> Tel. +41 (0)44 634 49 73
>> Fax: +41 (0)44 634 43 86
>> URL : http://www.biostat.uzh.ch/aboutus/people/sabanes.html
>>
>> _______________________________________________
>> R-sig-hpc mailing list
>> R-sig-hpc at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>