[R] parallel processing in r...

[Jeff Newmiller]

Use "top" at the bash prompt.

Read about the "mc.cores" parameter to mclapply.

Make a simplified example version of your analysis and post your question in the context of that example [1][2][3]. You will learn about the issues you are dealing with in the process of trimming your problem, and will have code you can share that demonstrates the issue without exposing private information.

Running parallel does not necessarily improve performance because other factors like task switching overhead and Inter-process-communication (data sharing) can drag it down. Read about the real benefits and drawbacks of parallelism... there are many discussions out there out there... you might start with [4].


[1] http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example

[2] http://adv-r.had.co.nz/Reproducibility.html

[3] https://cran.r-project.org/web/packages/reprex/index.html (read the vignette) 

[4] https://nceas.github.io/oss-lessons/parallel-computing-in-r/parallel-computing-in-r.html

On June 30, 2018 10:07:49 AM PDT, akshay kulkarni <akshay_e4 using hotmail.com> wrote:
>dear members,
>I am using mclapply to parallelize my code. I am using Red Hat Linux in
>AWS.
>
>When I use mclapply, I see no speed increase. I doubt that the Linux OS
>is allowing fewer than the maximum number of cores to mclapply ( by
>default, mclapply takes all the available cores to it).
>
>How do you check if the number of workers is less than the output given
>by detectCores(), in Linux? Is there any R function for it?
>
>I do acknowledge that help on an OS is not suitable for this mailing
>list, but even Internet could'nt help me. Therefore this mail......
>
>very many thanks for your time  and effort...
>yours sincerely,
>AKSHAY M KULKARNI
>
>	[[alternative HTML version deleted]]
>
>______________________________________________
>R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

-- 
Sent from my phone. Please excuse my brevity.