[Bioc-devel] rtracklayer BUG: `export(x, path, index=TRUE)` appears not to close filehandle on tabix files produced

Cook, Malcolm MEC at stowers.org
Thu Nov 8 23:32:48 CET 2012

rtracklayer developers (Michael/Vincent/Robert),

I find that tabix indexed exporting too many bed files causes an error.

The session following my signature reproduces the error.  

It provides sessionInfo() details prior to the code causing the error because sessionInfo() FAILS with 'too many open files' after running this code (as does anything the opens files).

The error does NOT occur when index=FALSE.  Only when index=TRUE.

I expect that the tabix calls are not cleaning up open file handles correctly.

uname -a tells me on my mac OSX that I can have 256 files open.  

The bug happens during the 253rd bedfile. 

openConnections() returns nothing.  

closeAllConnections() does not clean them up.

lsof to list open files at the command line does NOT show them.

Michael(?), you resolved a similar issue I once reported with rtracklayer when creating bigBed files : https://lists.soe.ucsc.edu/pipermail/genome/2012-February/028343.html 

Any suggestions for workarounds?  Any possibility of a quick patch to released rtracklayer?

Thanks for rtracklayer!

~Malcolm Cook

bash-3.2$ R

R version 2.15.1 (2012-06-22) -- "Roasted Marshmallows"
Copyright (C) 2012 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(rtracklayer)
Loading required package: GenomicRanges
Loading required package: BiocGenerics

Attaching package: 'BiocGenerics'

The following object(s) are masked from 'package:stats':


The following object(s) are masked from 'package:base':

    Filter, Find, Map, Position, Reduce, anyDuplicated, cbind, colnames, duplicated, eval, get, intersect, lapply, mapply, mget, order, paste, pmax, pmax.int, pmin, pmin.int, rbind, rep.int, rownames, sapply, setdiff, table, tapply, union, unique

Loading required package: IRanges
Warning message:
package 'GenomicRanges' was built under R version 2.15.2 

> sessionInfo()
R version 2.15.1 (2012-06-22)
Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)

[1] C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rtracklayer_1.18.0   GenomicRanges_1.10.4 IRanges_1.16.4       BiocGenerics_0.4.0  

loaded via a namespace (and not attached):
 [1] BSgenome_1.26.1   Biostrings_2.26.2 RCurl_1.95-3      Rsamtools_1.10.1  XML_3.95-0.1      bitops_1.0-4.2    parallel_2.15.1   stats4_2.15.1     tools_2.15.1      zlibbioc_1.4.0   

> x<-sapply(sprintf('deleteme_%s.bed',1:1000), function(conn) {export(GRanges('X',IRanges(1,2)),conn,index=TRUE);1})
Error in value[[3L]](cond) : index build failed
  file: /Volumes/SAN1/Users/mec/deleteme/253.bed.gz
In addition: Warning message:
In doTryCatch(return(expr), name, parentenv, handler) :
  [ti_index_build2] fail to create the index file.

> sessionInfo()
Error in gzfile(file, "rb") : cannot open the connection
In addition: Warning message:
In gzfile(file, "rb") :
  cannot open compressed file '/Library/Frameworks/R.framework/Versions/2.15/Resources/library/rtracklayer/Meta/package.rds', probable reason 'Too many open files'

More information about the Bioc-devel mailing list