Thank you Steve for your time.

While I would like to say I understand you explanation, facts would indicate
I'm doing something wrong. I'm not sure what, because I've tried identical
approach on another function where it works (using .export), but with a
warning that I've already exported some variables. I use doSMP (neither doMC
(testing windows support) nor doSNOW work for me).

To clear up where variables are located. I have a wrapper function that
supplies arguments that are passed to inline functions. Perhaps this will be
more readable:

wrapperFunction <- function(..., effect.distance) {
   ...
   makeDistance <- function(..., effect.distance) {...}

   foreach(..., .export("effect.distance")) %dopar%
      makeDistances(..., effect.distance = effect.distance)
   ...
}

What is confusing me is that the there is already a warning that variables (
effect.distance and rst.raw) have already been exported, yet worker(s) can't
find it. As mentioned in my previous email, single run (%do%) goes through
smoothly.

> foreach(i.xy = xy.list, i.cnt = icount(),
     .export = c("effect.distance", "rst.raw")) %dopar% makeDistances(xy =
i.xy, rst.raw, effect.distance, cnt = i.cnt)

Error in makeDistances(xy = i.xy, rst.raw, effect.distance, cnt = i.cnt) :
  task 2 failed - "object 'effect.distance' not found"
In addition: Warning message:
already exporting variable(s): effect.distance, rst.raw


Cheers,
Roman




On Tue, Aug 24, 2010 at 3:16 PM, Stephen Weston
<stephen.b.weston@gmail.com>wrote:

> Roman,
>
> When %dopar% is executed with certain parallel backends, it analyzes
> the right hand expression to determine if there are any variables that need
> to be exported.  If a symbol is referenced in the expression that is
> defined
> in the enclosing environment, it will export that variable to the parallel
> workers.  If it is defined in the global environment, or in some package,
> it will not automatically export it.
>
> You don't show where "effect.distance" is defined.  If it is defined
> in the global environment, you could explicitly export it via the foreach
> ".export" argument.  See the foreach man page for details.  If it is
> defined in a package, you could use the ".packages" argument to
> load that package on each of the parallel workers.
>
> Note that this process of automatically exporting variables is performed
> in doSNOW and doMPI, but not in doMC.  That is because the multicore
> package uses the "fork" system call to create the worker processes.
> Processes that are forked get a copy of the parent process's execution
> environment, and so variables don't generally need to be exported,
> either implicitly or explicitly.  Thus, you could run into problems with
> undefined variables with doSNOW, even though your program runs fine
> with doMC.
>
> - Steve
>
>
>
> On Tue, Aug 24, 2010 at 8:13 AM, Roman Luštrik <roman.lustrik@gmail.com>
> wrote:
> > Dear list,
> >
> > I once again seek your advice in understanding the following.
> > makeDistances is a function I feed to foreach. The computation intensive
> > part is done by a second function calcDistance.
> >
> >    # Run in parallel
> >    makeDistances <- function(xy, rst, effect.distance, cnt, num.points) {
> >        source("d:/workspace/Texas Ranger/calcDistance.R")
> >        for (j in 1:num.points) {
> >            file.name <- paste("walker", cnt, "_", j, sep = "")
> >            clc <- calcDistance(object = rst, origin.point = xy[j,],
> >                      effect.distance = effect.distance)
> >            writeRaster(clc, file.name, overwrite = TRUE)
> >        }
> >    }
> >
> >    foreach(xy = xy.list, cnt = icount()) %do%
> >       makeDistances(xy, rst = rst, effect.distance = effect.distance,
> >       cnt, num.points = dim(xy.list[[cnt]])[1])
> >
> > When I run makeDistances via %do% things work out fine, but I'm unable to
> > run via %dopar%. The following error is returned:
> >
> > Error in makeDistances(xy, rst = rst, effect.distance = effect.distance,
>  :
> >  task 1 failed - "object 'effect.distance' not found"
> >
> >
> > It seems like a scoping issue, however I failed to comprehend how to feed
> > the argument to the function to make it work. Any advice will be much
> > appreciated.
> >
> >
> > Cheers,
> > Roman
> >
> > --
> > In God we trust, all others bring data.
> >
> >        [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-hpc mailing list
> > R-sig-hpc@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
> >
>



-- 
In God we trust, all others bring data.

	[[alternative HTML version deleted]]

