[BioC] AWS cluster not finding installed R packages

[John K DeHart]

Dan Tenenbaum <dtenenba at ...> writes:

> 
> On Mon, Jul 15, 2013 at 7:24 AM, John [guest] <guest at ...> wrote:
> >
> > I'm building an aws cluster with bioconductor and I have to install some 
packages from CRAN prior to the
> start of the run. I'm using doSNOW for the parallel tasks. When I start 
the run I receive the error... "X
> Package not found" where X  is my installed packages. If I run on a single 
multicore system this error is not received.
> >
> > I tried logging into each node and installing locally but I still 
receive the same error.
> >
> > Any ideas?
> 
> I think I need more information. What is the command or code you are
> using to install packages on all nodes in the cluster?
> 
> How exactly did you start up the cluster?
> 
> Dan
> 
> > Thanks
> >
> >  -- output of sessionInfo():
> >
> > "X Package not found"
> >
> > --
> > Sent via the guest posting facility at bioconductor.org.
> >
> > _______________________________________________
> > Bioconductor mailing list
> > Bioconductor at ...
> > https://stat.ethz.ch/mailman/listinfo/bioconductor
> > Search the archives: 
http://news.gmane.org/gmane.science.biology.informatics.conductor
> 
> _______________________________________________
> Bioconductor mailing list
> Bioconductor at ...
> https://stat.ethz.ch/mailman/listinfo/bioconductor
> Search the archives: 
http://news.gmane.org/gmane.science.biology.informatics.conductor
> 
> 

Dan,

I'm starting the system using the link "Start parallel Cluster with ssh 
access" on the Bioconductor - Cloud AMI web page. I create the cluster in R 
with the "cl <- makePSOCKcluster" as specified on the same page (by reading 
the host file...). The cluster starts normally and I can see that is 
contains all of the nodes. 

I'm exporting the libraries and data:
clusterEvalQ(cl, library(the library I need))
clusterExport(cl, c('data', 'i', 'need'))
registerDoSNOW(cl)

When I execute I get the error "The library X is not found". If I run this 
on the local cores only... no problem.

I did try and ssh into each node and install the library locally then went 
back to the master to execute and the same thing happened.

Thanks for the help... I really appreciate it,
John