[R-sig-hpc] Creating different working directories for each node?

[Novack-Gottshall, Philip M.]

To Brian and anyone else following along on this thread:

I wanted to reply with a solution. (Thanks also to Luke Tierney for
feedback.)

The simplest solution seems to be to take advantage of the temporary
directories automatically populated by R / snow / snowfall when the
cluster is initialized and then use these directories to store the
temporary files. Because each tempdir is unique to each process, each
worker by definition is assigned its own tempdir.

In my case, I just needed to add the following into my function:

testfun <- function(...) {
     # run normal code [snipped out here]

     # switch to temporary directory for function that writes/scans temp
file
     wd <- getwd()
     td <- tempdir()
     setwd(td)
     out <- filemakingfunction(...)

     # return to normal working directory
     setwd(wd)

     return(out)
     }

This seems to work without any errors when called within clusterApply()
and clusterApplyLB().

The fix is confirmed by the following: If 'td' above is assigned the
shared 'wd' instead of 'tempdir()', I get nonsensical results for the
portions of the analyses using the temp file.

Thanks again to everyone who provided help!

Phil




On 8/12/2014 9:57 AM, Novack-Gottshall, Philip M. wrote:
> The tempfile option might work, as might using Sys.info()["nodename"] to
> specify individual subdirectories.
>
> Are you aware of any functions that can be used to identify which CPU a
> process has been spawned to? The package 'snowfall' has sfGetCluster(),
> but that doesn't seem to be able to be called *within* a cluster function.
>
> Thanks again,
> Phil
>
>
> On 8/11/2014 4:10 PM, Brian G. Peterson wrote:
>> You didn't tell us which clusterning mechanism you're using, but most 
>> will allow you to retrieve a unique node id or node number, which you 
>> could use to create a subdirectory.
>>
>> Also, 'tempfile' in R will create a vector of unique temporary file 
>> names, which you could pass to the nodes, or append with a node id.
>>
>> Brian
>>
>> On 08/11/2014 03:59 PM, Novack-Gottshall, Philip M. wrote:
>>> Greetings,
>>>
>>> I'm trying to run some code on a cluster in which an internal qhull
>>> convex-hull function repeatedly writes then scans a dummy file. (Very
>>> inefficient, I know, but c'est la vie.) The problem is that all nodes
>>> share the same default working directory, and get confused because they
>>> are all trying to read/write the same dummy file. (So far as I know,
>>> there is no way to specify unique file names for the dummy files in the
>>> internal .C function, especially with my lack of C prowess.)
>>>
>>> I've been playing around with having my function (called using
>>> snowfall's 'sfClusterApply') specify unique working directories (using
>>> 'setwd'), say one wd for each CPU, so that the individual dummy files
>>> are set within unique directories. This seems a plausible wrap-around so
>>> long as I confirm that I'm matching up the wd with the correct CPU process.
>>>
>>> I wanted to check whether anyone has any other recommendations before I
>>> waste my time on further troubleshooting.
>>>
>>> If it's relevant, I'm running my code using management package snowfall
>>> on a CentOS/OpenMPI Intel cluster with 2 hyperthreaded 6-cores in each
>>> of 16 nodes (allowing 382 functional CPUs/unique working directories for
>>> the job).
>>>
>>> Thanks,
>>> Phil
>>>
>> _______________________________________________
>> R-sig-hpc mailing list
>> R-sig-hpc at r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
>>
>>

-- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Phil Novack-Gottshall
 Associate Professor
 Department of Biological Sciences
 Benedictine University
 5700 College Road 
 Lisle, IL 60532

 pnovack-gottshall at ben.edu
 Phone: 630-829-6514
 Fax: 630-829-6547
 Office: 332 Birck Hall
 Lab: 107 Birck Hall
 http://www1.ben.edu/faculty/pnovack-gottshall
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~