[R] Parallel runs of an external executable with snow in local

[Uwe Ligges]

On 06.08.2012 17:07, Xavier Portell/UPC wrote:
> Thanks Uwe but, actually, I did so.
>
> Since “filetorun.exe” looks in the current folder for “input.txt”, I tried moving all needed files to a newly created temporary folder “tmp.id” (say, tmp.1) and running the executable. This works fine by doing it directly from the windows command line but not by doing it from R, since using:
>
> #
> System(“C:/Users/…/currentworkdirectory/temp.1/filetorun.exe”)
> #
>
> Causes “filetorun.exe” to look for “input.txt” in “C:/Users/…/currentworkdirectory”. So there’s no point on moving files to a folder, it seems that input file must be situated in the current R work directory. Does anybody know how to avoid this behaviour?

Errr, you mean you cannot specify an input for "filetorun.exe"?

If so, you can ask R to switch to use another working directory for each 
instance.

Best,
Uwe Ligges







>
> I hope I’ve explained that clearly,
>
> Xavier Portell Canal, PhD candidate.
> Department of Agri-food Engineering,
> Universitat Politècnica de Catalunya
>
> -----Uwe Ligges <ligges at statistik.tu-dortmund.de> ha escrit: -----
>
> Per a: Xavier Portell/UPC <xavier.portell at upc.edu>
> De: Uwe Ligges <ligges at statistik.tu-dortmund.de>
> Data: 05/08/2012 07:46PM
> a/c: r-help at r-project.org
> Assumpte: Re: [R] Parallel runs of an external executable with snow in local
>
>
>
> On 03.08.2012 19:21, Xavier Portell/UPC wrote:
>> Hi everyone,
>>
>> I'm aiming to run an external executable (say filetorun.EXE) in parallel. The external executable collect needed data from a file, say "input.txt" and, in turn,generates several output files, say "output.txt". I need to generate "input.txt", run the executable and keep "input.txt" and "output.txt". I'm using Windows 7, R version 2.15.1 (2012-06-22) on RStudio and platform: i386.pc.mingw32/i386 (32-bit).
>>
>> My first attempt was a R code which, by using
>>     System("filetorun.EXE", intern = F, ignore.stdout = F,
>>             ignore.stderr = F, wait = T, input = NULL,
>>             show.output.on.console = T, minimized = F, invisible = T))
>> , ran the executable and kept required files to a conveniently named folder. After that I changed my previous R script so I could use the function lapply().This script apparently worked fine.
>>
>> Finally, I tried to parallelize the problem by using snow and parLapply(). The resulting script looks like this:
>>
>> ## Not run
>> #
>> library(snow)cl <- makeCluster(3, type = "SOCK")
>> clusterExport(cl,list('param.esp','copy.files','for12.template','program.executor'))
>> parLapply(cl,a.list,a.function))stopCluster(cl)
>> #
>> ##End not run
>>
>> Although it runs, the parallelized version is messing up the input parameters to pass to the executable (see table below, where parameters P1 and P2 are considered. ".s" comes from the serial code and ".p" from the parallelized one):
>>     s r P1.s P2.s P1.p P2.p
>> 1 1 1  1.0 3.00  2.0 3.00
>> 2 2 1  1.5 3.00  2.0 3.75
>> 3 3 1  2.0 3.00  2.0 3.00
>> 4 4 1  1.0 3.75  1.5 3.00
>> 5 5 1  1.5 3.75  1.5 3.00
>> 6 6 1  2.0 3.75  2.0 3.75
>>
>> My first thought to avoid the described behaviour was creating a temporary file, say "tmp.id" with id being an identification run number, and copying "filetorun.EXE" and "Input.txt" to "tmp.id". However, while doing so, I realised that although running the correct "filetorun.EXE" copy (i.e., the one in "tmp.id") R looks for "input.txt" in the work directory.
>
>
> Not sure about the real setup, but you can actually specify the path,
> not only filenames.
>
> Uwe Ligges
>
>
>>
>>
>> I've been looking thoroughly for a solution but I got nothing.
>>
>> Thanks for any help in advance,
>>
>>
>> Xavier Portell Canal
>>
>> PhD candidate
>> Department of Agri-food engineering,
>> Universitat Politècnica de Catalunya
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>