[BioC] AWS cluster not finding installed R packages

[Dan Tenenbaum]

On Fri, Jul 19, 2013 at 11:45 AM, john <john at enginetix.com> wrote:
> Thanks for the help.....
>
> I'm using bioc 2.11 (link to start a parallel cluster with ssh on the aws
> web page). Is there a process for installing optional R packages from CRAN?
>
> Sorry for the lack of info.... I'm still coming up to speed....
>

Please keep the bioconductor list in the loop so all can benefit from
the discussion.

Yes, you can use biocLite() or install.packages() to install packages from CRAN.

You will also need to install required packages on each cluster node;
the method for doing this varies with the different ways there are to
start a cluster, but with doSNOW, assuming you've already started a
cluster called 'cl', and the packages you want to install are survival
and abc,  you'd do something like:

clusterEvalQ(cl, {library(BiocInstaller); biocLite(c("survival", "abc"))})

Then you can run your parallel computation.
Dan



>
> Sent from my Verizon Wireless Phone
>
>
> ----- Reply message -----
> From: "Dan Tenenbaum" <dtenenba at fhcrc.org>
> To: <john at enginetix.com>
> Cc: "John [guest]" <guest at bioconductor.org>, "bioconductor at r-project.org"
> <bioconductor at r-project.org>
> Subject: [BioC] AWS cluster not finding installed R packages
> Date: Fri, Jul 19, 2013 1:58 pm
>
>
> On Fri, Jul 19, 2013 at 10:56 AM,  <john at enginetix.com> wrote:
>> Dan,
>>
>> I'm just using the standard bioconductor aws ami and I'm installing some
>> extra packages from CRAN. Working from R when I'm running a doSNOW
>> cluster.
>> I see that all of the nodes show up in the cluster, and I'm exporting the
>> libraries as required. Unfortunately, when I run a job I receive an error
>> that the packages are not found (it works if I use only local nodes). I
>> have
>> tried to load the required packages on each node... still no luck....
>>
>
> Again, difficult to say what is going on without more information,
> such as a full transcript of your session. Also which AMI are you
> starting (which BioC version?).
>
> Dan
>
>
>> Thank,
>> John
>>
>> -------- Original Message --------
>> Subject: Re: [BioC] AWS cluster not finding installed R packages
>> From: Dan Tenenbaum <dtenenba at fhcrc.org>
>> Date: Mon, July 15, 2013 7:37 am
>> To: "John [guest]" <guest at bioconductor.org>
>> Cc: "bioconductor at r-project.org" <bioconductor at r-project.org>,
>> john at enginetix.com
>>
>> On Mon, Jul 15, 2013 at 7:24 AM, John [guest] <guest at bioconductor.org>
>> wrote:
>>>
>>> I'm building an aws cluster with bioconductor and I have to install some
>>> packages from CRAN prior to the start of the run. I'm using doSNOW for
>>> the
>>> parallel tasks. When I start the run I receive the error... "X Package
>>> not
>>> found" where X is my installed packages. If I run on a single multicore
>>> system this error is not received.
>>>
>>> I tried logging into each node and installing locally but I still receive
>>> the same error.
>>>
>>> Any ideas?
>>
>> I think I need more information. What is the command or code you are
>> using to install packages on all nodes in the cluster?
>>
>> How exactly did you start up the cluster?
>>
>> Dan
>>
>>
>>> Thanks
>>>
>>> -- output of sessionInfo():
>>>
>>> "X Package not found"
>>>
>>> --
>>> Sent via the guest posting facility at bioconductor.org.
>>>
>>> _______________________________________________
>>> Bioconductor mailing list
>>> Bioconductor at r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/bioconductor
>>> Search the archives:
>>> http://news.gmane.org/gmane.science.biology.informatics.conductor
>