[R-sig-hpc] speed up a function containing resampling

[John Denton]

Hi, all.

I’m trying to assemble a set of functions to do a resampling of a value that is calculated by a combination of subsampling and sums. Each iteration involves reading in a number of large files, sampling from the entries in the files, and doing calculations based on these subsamples. The way I have it set up right now is through a combination of lapply and for. The outer resampling is then done with mclapply.

I have two problems. First, the code used to run, but now doesn’t. I get the error

In parallel:::mclapply(iters, do.call(tii.calc, config), mc.cores = ncores) :
  all scheduled cores encountered errors in user code
>

Any ideas on how to fix this problem? I am running R 3.3.1 in the terminal on a personal OS X Yosemite machine.


Second, I’m hoping to speed up the function. It is currently very slow, and I need to do a number of outer resamplings in the 100s or 1000s.

Here is the outer resampling code that calls the function itself (attached):

source("tree_collapser.R")
source("~/taxon_influence/tii_calc.r")
library(phangorn)
library(SnowballC)
library(ape)
library(stringr)

outgroup <- "Hemichordata"
burnin <- 0.5
n.iters <- 2
ncores <- parallel:::detectCores()/4

d <- read.nexus.data(as.character("tully.nex"))
nom <- sort(names(d)[-which(names(d) == outgroup)])

miltrees <- list.files(pattern="^minus[A-Z].*\\.t$")
fulltree <- read.nexus("tully.nex.t_alltaxa")
##iter.list <- rep(list(miltrees), n.iters)
iters <- 1:n.iters

##tii.rep <- parallel:::mclapply(iters, tii.calc(iter.list, taxnames=nom,
## full.trees=fulltree, outgroup=outgroup, burnin=burnin), mc.cores=ncores)


config <- list(treelist=miltrees, taxnames=nom, full.trees=fulltree,
outgroup=outgroup, burnin=burnin)

tii.rep <- parallel:::mclapply(iters, do.call(tii.calc, config), mc.cores=ncores)

Thanks!

~John


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://stat.ethz.ch/pipermail/r-sig-hpc/attachments/20170503/ed71ec60/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tii_calc.r
Type: application/octet-stream
Size: 1937 bytes
Desc: tii_calc.r
URL: <https://stat.ethz.ch/pipermail/r-sig-hpc/attachments/20170503/ed71ec60/attachment.obj>