[Bioc-devel] Redirect workers output to STDERR, how to do so with current BiocParallel::SnowParam()?

Leonardo Collado Torres lcollado at jhu.edu
Thu May 21 16:58:04 CEST 2015


This might be a BioC support website question, but maybe it's a bug.

In previous versions of BiocParallel, you could specify where to
direct the output from the workers by using the 'outfile' argument.
For example, SnowParam(outfile = Sys.getenv('SGE_STDERR_PATH')). I'm
not finding how to do so with the current version (1.3.12).

I understand that SnowParam() has a ... argument that gets passed to
snow::makeCluster(), according to the SnowParam() docs. I also see
that when log = TRUE, a script is used instead of snow. But log =
FALSE by default.

If I use snow, the 'outfile' argument does work as shown below:

## Nothing gets printed by default
> cl <- makeCluster(2, type = 'SOCK')
> y <- clusterApply(cl, 1:2, print)

## Use outfile now, print works
> cl <- makeCluster(2, type = 'SOCK', outfile = NULL)
starting worker for localhost:11671
starting worker for localhost:11671
> y <- clusterApply(cl, 1:2, print)
Type: EXEC
Type: EXEC
[1] 1
[1] 2

> packageVersion('snow')
[1] ‘0.3.13’

However, I can't use 'outfile' with SnowParam:

## SerialParam works
> x <- bplapply(1:2, print, BPPARAM = SerialParam())
[1] 1
[1] 2

## No printing by default with SnowParam, as expected
> x <- bplapply(1:2, print, BPPARAM = SnowParam(workers = 1))

## Can't pass 'outfile' argument
> x <- bplapply(1:2, print, BPPARAM = SnowParam(workers = 1, outfile = NULL))
Error in bplapply(1:2, print, BPPARAM = SnowParam(workers = 1, outfile
= NULL)) :
  error in evaluating the argument 'BPPARAM' in selecting a method for
function 'bplapply': Error in envRefSetField(.Object, field, classDef,
selfEnv, elements[[field]]) :
  ‘outfile’ is not a field in class “SnowParam”

Digging at the code, I see that in SnowParam() the ... argument is
saved in .clusterargs

args <- c(list(spec = workers, type = type), list(...))
    .clusterargs <- lapply(args, force)

However, ... is still passed to .SnowParam(), and .SnowParam() fields are:


So, I'm wondering if ... should not be passed to .SnowParam().

## Trying to pass outfile to .clusterargs directly doesn't work
## Actually, I'm surprised it didn't crash .SnowParam()
> x <- bplapply(1:2, print, BPPARAM = SnowParam(workers = 1, .clusterargs=list(outfile = NULL)))

Full log: https://gist.github.com/9ac957c6cad1c07f4ea4


More information about the Bioc-devel mailing list