[R] foreach not recognizing functions in memory

[Bos, Roger]

My apologies for not mentioning which backend and OS I am using. I am using doSMP on a windows machine.  I read up on doSMP, but all the examples they show are based on calling foreach directly, not from within a function.

The code works fine using %do% instead of %dopar%.  Does anyone have an example where user written function has to be exported to slave processors?

Thanks,

Roger 

-----Original Message-----
From: wootten.adrienne at gmail.com [mailto:wootten.adrienne at gmail.com] On Behalf Of Adrienne Wootten
Sent: Tuesday, July 12, 2011 5:49 PM
To: Bos, Roger
Cc: r-help at r-project.org
Subject: Re: [R] foreach not recognizing functions in memory

Roger,

I think I know why you are having a problem.  Your foreach is using %dopar% and judging from your output you have active parallel processing.  When running something in paralell, the master is the processor used by the Rgui, and the slaves are in the background.  My guess is the the function you made is the memory of the master, but not in the memory for the slaves.  You need to pass the function "TestFun" to your slaves.  If the slave don't know about it, using foreach with %dopar% will fail since the function doesn't exist on the slave processors.  Without knowing what package you used to do your parallel processing I can't tell you how to pass the function you created to your slave processors.

Try running the foreach command with %do% in place of %dopar%.  If the command works, then it is probably a problem that function isn't in memory on the slave processors.

Adrienne

On Tue, Jul 12, 2011 at 5:28 PM, Bos, Roger <roger.bos at rothschild.com> wrote:
> All,
>
> I am not understanding the scoping used in foreach when it is used 
> inside a function.  I keep getting "could not find function" errors 
> for functions that are in memory when I try to use foreach within a 
> function call.  I have a simple example below.  "testFun" is in memory 
> and works when called by foreach directly, but when I place foreach in 
> a function called "helper" and then call helper, foreach cannot find "testFun".
> Can anyone provide a fix?  I hope its just a simple mistake on my part 
> and not a limitation of foreach.
>
>> source(.trPaths[5], echo=TRUE, max.deparse.length=150)
>
>> "testFun" <- function(m) {
> +     out <- m*m
> + }
>
>> out <- foreach(m=1:10, .combine=rbind) %dopar% testFun(m)
>
>> "helper" <- function(i) {
> +
> +     out <- foreach(m=1:i, .combine=rbind) %dopar% testFun(m) }
>
>> out <- helper(10)
> Error in testFun(m) : task 1 failed - "could not find function 
> "testFun""
>>
>
> Thanks,
>
> Roger
>
> ***************************************************************
>
> This message is for the named person's use only. It 
> may\...{{dropped:20}}
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



--
Adrienne Wootten
Graduate Research Assistant
State Climate Office of North Carolina
Department of Marine, Earth and Atmospheric Sciences North Carolina State University
***************************************************************

This message is for the named person's use only. It may
contain confidential, proprietary or legally privileged
information. No right to confidential or privileged treatment
of this message is waived or lost by an error in transmission.
If you have received this message in error, please immediately
notify the the sender by e-mail, delete the message and all 
copies from your system and destroy any hard copies.  You must
not, directly or indirectly, use, disclose, distribute, 
print or copy any part of this message if you are not
the intended recipient.

****************************************************************



______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email