[BioC] EBS volumes with the Bioconductor AMI: how to change default behaviour

[Dan Tenenbaum]

On Mon, Aug 15, 2011 at 1:40 AM, Quin Wills <qilin at quinwills.net> wrote:
> Thanks Dan
> I did think it slightly unusual that so many volumes are created and stay
> attached. I look forward to heard from you!


BTW, this no longer happens. Be sure and use the latest AMI ID from:
http://bioconductor.org/help/bioconductor-cloud-ami/

Thanks,
Dan


> If you do have any advice on what you think would be the most appropriate
> way for R to terminate an instance once a job is done, I'd also appreciate
> it enormously.
> Thanks,
> Quin
>
>
> On 12 August 2011 18:46, Dan Tenenbaum <dtenenba at fhcrc.org> wrote:
>>
>> On Fri, Aug 12, 2011 at 12:32 AM, Quin Wills <qilin at quinwills.net> wrote:
>> > Thanks for the advice Dan.
>> >
>> > The reason I like to use S3 is that I like to run jobs, log out and
>> > have them automatically
>> > shut down when done. At the moment I'm just running the following
>> > function for
>> > automated shutdown of my instances from within my R script:
>> >
>> > shutdown <- function(time=0) return(system(paste("echo 'sudo halt' |
>> > at now + ",time," min",sep="")))
>> >
>> > Even if I set my instance's shutdown behaviour to "terminate" (in the
>> > AWS management console),
>> > those EBS volumes seem to persist when I automate termination this way.
>>
>> Hi,
>>
>> I just tried running the Bioconductor AMI as a different user and I
>> notice that those volumes are created and stay "attached" to the
>> instance even after the instance is terminated. I'm not sure why they
>> are created in the first place; I'll look into that and report back to
>> you. You can safely detach and delete them after your instance is
>> halted. You can do that in the EC2 web console, with EC2 command-line
>> tools, or calling the EC2 API from a programming language. I can send
>> you an example of the latter if you are interested.
>>
>>
>> >
>> > Do you perhaps have a recommendation on how better to make sure my
>> > instance shuts
>> > down once the job is done? Ideally it would be great if it could fire
>> > off a quick
>> > email too, but this doesn't seem so easy to do unless I create my own
>> > AMI I think.
>> >
>>
>> You might look into Amazon's Simple Email Service.
>> http://aws.amazon.com/ses/
>>
>> Dan
>>
>>
>> > Thanks a ton,
>> > Quin
>> >
>> >
>> >>>On Thu, Aug 11, 2011 at 6:11 AM, Quin Wills <qilin at quinwills.net>
>> >>> wrote:
>> >>> Hello Bioconductor AMI gurus
>> >>>
>> >>> Delighted that Bioconductor has an AMI with pre-loaded bells and
>> >>> whistles.
>> >>> I'm hardly an AWS guru (yet?), and in particular feel like all the
>> >>> dots
>> >>> aren't connecting in my brain regarding EBS.
>> >>>
>> >>> So I see that the Bioconductor AMI automatically initiates 1 x 20GiB
>> >>> root
>> >>> EBS volume, and 3 x 30 GiB extra volumes, correct?
>> >>> What if I don't want
>> >>> these? Presumably just detaching and deleting them in the AWS
>> >>> management
>> >>> console is one way to do it? Is this the only (reasonably easy) way?
>> >>
>> >>
>> >>The AMI "lives" on these EBS volumes so you don't want to delete them.
>> >>You may find you don't even own them.
>> >>
>> >>
>> >>
>> >>> For the moment I'm just using AWS for CPU-intensive work that I need
>> >>> to
>> >>> speed up. I have an S3 bucket and am using the omegahat RAmazonS3
>> >>> library to
>> >>> access and save data on a semi-permanent basis. Does this seem like a
>> >>> reasonable tactic? For the moment, the sizes of the data objects in my
>> >>> S3
>> >>> bucket are manageable.
>> >>
>> >>If it works for you, it is reasonable. The reason we don't use S3 is
>> >>that we find it slow, plus it is a two-step process to push files to
>> >>S3 from your AMI, then pull them from S3 to your local machine, as
>> >>opposed to using scp to copy files directly in one step.
>> >>
>> >>But if you find that S3 works for you, there's no reason not to use it.
>> >>Dan
>> >>
>> >>> Perhaps there's a link to an idiots guide on "EBS vs S3" options and
>> >>> suggestions when using the Bioconductor AMI?
>> >>>
>> >>> Thanks in advance for any wisdom,
>> >>> Quin
>> >
>> > _______________________________________________
>> > Bioconductor mailing list
>> > Bioconductor at r-project.org
>> > https://stat.ethz.ch/mailman/listinfo/bioconductor
>> > Search the archives:
>> > http://news.gmane.org/gmane.science.biology.informatics.conductor
>> >
>
>
>
> --
>
> Quin Wills
> Live the kind of life that when you die half the world mourns the loss of a
> great sentience, whilst the other half are just grateful that it's over.
>
> Brasenose College
> Oxford
> OX1 4AJ
> tel: +44 (0)7951 335 714
> inet: www.quinwills.net
>