I think you should try increasing the cores (for the tally step). The
memory consumption in that step is mostly due to the summaries. All the
summaries need to be generated prior to the filtering, so you might as well
calculate them more quickly. Another idea is to parallelize at a higher
level, so that each core is responsible for both tallying and filtering
(for, say, one chromosome). This would be more work on your part, though.
Eventually VariantTools needs to make this easier.

Michael


On Thu, Jun 13, 2013 at 2:02 PM, Eric Foss [guest]
<guest@bioconductor.org>wrote:

>
> I am running VariantTools on 40 not particularly large bam files (~700
> Mb). When running 5 in parallel on the same machine (12 cores, 48 Gig RAM),
> each running on a single core, it took several hours for the first one to
> finish, and the other 4 are still running. In total, the remaing 4 are
> taking up 70% of the RAM. Does anyone have suggestions for how to run
> VariantTools for optimum speed? I can increase the number of cores, but
> then I have to run fewer files because otherwise I run out of memory. And
> if I run 5 at a time each with a single core, I'm coming close to using up
> all the memory, and I don't know if that will really slow things down and
> if it would therefore be better to run just 4 at a time. Any suggestions
> are appreciated.
>
> Thank you.
>
> Eric
>
>  -- output of sessionInfo():
>
> These are running in batch mode so I don't have that information.
>
> --
> Sent via the guest posting facility at bioconductor.org.
>
> _______________________________________________
> Bioconductor mailing list
> Bioconductor@r-project.org
> https://stat.ethz.ch/mailman/listinfo/bioconductor
> Search the archives:
> http://news.gmane.org/gmane.science.biology.informatics.conductor
>

	[[alternative HTML version deleted]]

