Here I just want to know why cannot I use .GlobalEnv$y to access y in parLapply?
parLapply does not attempt to make the GlobalEnv's of all the subprocesses identical.
Many times that would not be appropriate, either because you want the various workers
to do different things or because it would take too much time.  My trick worked because
it appears to send the function FUN to the workers along with FUN's environment and
its ancestral environments, up to but not including .GlobalEnv, the same as the save() function
does.

By the way, my function should have a parent= argument in its call to list2env()
so its own environment (which includes 'FUN' and '...') doesn't get added to FUN's
environment list.
  withGlobals <- function(FUN, ..., envir=parent.frame()){
       environment(FUN) <- list2env(list(...), parent=envir)
       FUN
  }

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com

From: Yu Wan [mailto:walterwan@126.com]
Sent: Friday, December 27, 2013 3:46 AM
To: William Dunlap; alexios@4dscape.com
Cc: r-devel@r-project.org
Subject: Re: [Rd] Parallel computing: how to transmit multiple parameters to a function in parLapply?

Thanks to William and Alexios, both of your solution worked well and solved my problem.

Here I just want to know why cannot I use .GlobalEnv$y to access y in parLapply?

Best regards,

Yu

On 24/12/2013 11:20 PM, alexios ghalanos wrote:

This works:



clusterExport(cl, c("f","y"), envir=environment())

r <- parLapply(cl, x, function(x) f(x,y))



You need to export your function ("f") and additional variables ("y"), and then

define that function inside parLapply ("f(x,y)"). If you were to also make use of

additional libraries (or source some scripts) then you should also consult

"clusterEvalQ".

The makeCluster command (at least in windows via socket) just initializes new R

processes which do not know about your functions or variables unless you

export those to them.



Perhaps a question best suited for R-help.



Alexios

On 25/12/2013 2:57 AM, William Dunlap wrote:

You can put the function of interest and any global

variables it needs into a private environment, which gets sent

along with the function to the child processes.  E.g.



library(parallel)

cl3 <- makeCluster(3)

y <- c(1,100,10000)

addY <- function(x) x + y

withGlobals <- function(FUN, ...){

    environment(FUN) <- list2env(list(...))

    FUN

}

parLapply(cl3, 1:4, withGlobals(addY, y=y))

# [[1]]

# [1]     2   101 10001

#

# [[2]]

# [1]     3   102 10002

# ...



Bill Dunlap

Spotfire, TIBCO Software

wdunlap tibco.com





-----Original Message-----

From: r-devel-bounces@r-project.org<mailto:r-devel-bounces@r-project.org> [mailto:r-devel-bounces@r-project.org] On Behalf

Of Yu Wan

Sent: Monday, December 23, 2013 10:16 PM

To: r-devel@r-project.org<mailto:r-devel@r-project.org>

Subject: [Rd] Parallel computing: how to transmit multiple parameters to a function in

parLapply?



Hi R-developers



In the package Parallel, the function parLapply(cl, x, f) seems to allow

transmission of only one parameter (x) to the function f. Hence in order to

compute f(x, y) parallelly, I had to define f(x, y) as f(x) and tried to

access y within the function, whereas y was defined outside of f(x).



Script:



library(parallel)



f <- function(x) {

  z <- 2 * x + .GlobalEnv$y  # Try to access y in the global scope.

  return(z)

}



np <- detectCores(logical = FALSE)  # Two cores of my laptop

x <- seq(1, 10, by = 1)

y <- 0.5  # Y may be an array in reality.

cl <- makeCluster(np)  # initiate the cluster

  r <- parLapply(cl, x, f)  # apply f to x for parallel computing

stopCluster(cl)



The r was a list with 10 empty elements which means f failed to access y.



Then I tested f without parallel computing:

z <- f(x)

print(z)

[1]  2.5  4.5  6.5  8.5 10.5 12.5 14.5 16.5 18.5 20.5



The results indicates that we can access y using .GlobalEnv$y in a function

without parLapply.



The question is, is there any method for me to transmit y to f, or access y

within f during parallel computing?



The version of my R is 3.0.1 and I am running it on a Win8-64x system.



Thanks,



Yu







--

View this message in context: http://r.789695.n4.nabble.com/Parallel-computing-how-

to-transmit-multiple-parameters-to-a-function-in-parLapply-tp4682667.html

Sent from the R devel mailing list archive at Nabble.com.



______________________________________________

R-devel@r-project.org<mailto:R-devel@r-project.org> mailing list

https://stat.ethz.ch/mailman/listinfo/r-devel


	[[alternative HTML version deleted]]

