[R] 'snow' package -- parallel process

[Prof Brian Ripley]

On 04/09/2013 18:57, Hui Du wrote:
>
>
> Hi R-community:
> I heard 'snow' package is a good tool to parallelize processes and speed them up. I tried to use it but was not successful. Could someboy point where I was wrong? Thanks.
> I want to read a HUGE file to R and hope 'snow' helps me to speed it up. Here are codes:

Why are you not using package 'parallel'?

But read the help for clusterApply: you have the arguments wrong.

And please do read the 'R Data Import/Export' manual and get read.delim 
working optimally first.

> library(snow)
>
> iFile = 'BIG.FILE.txt'
>
> numCluster = 4;
> readFile = function(file)
> {
>      orig_d = read.delim(file);
>      orig_d;
> }
> cl = makeCluster(numCluster, type = "SOCK");
> x = clusterApply(cl, readFile, iFile);
>
> I got the error
>
> Error in x[[i]] : object of type 'closure' is not subsettable
>
> I also tried to read multiple files once
>
> filenames = rep(iFile, numCluster);
> x = clusterApply(cl, readFile, filenames);
> stopCluster(cl);
>
> and got the same error
>
> Thanks you for your help.
> HXD
>
>
>
>
>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595