[Bioc-devel] problem in rbind with DelayedArray / HDF5Array package

[Michael Lawrence]

I'm surprised that DelayedMatrix inherits from DataTable. Originally
at least, DataTable was meant to represent a data.frame-like object,
not a matrix-like object. Since rbind2() and cbind2() assume that
DataFrame is a valid default implementation of DataTable, they will
coerce the matrix to a DataFrame, which is not going to end well. At
the very least, DelayedMatrix will need to override the rbind2() and
cbind2() methods with DataTable in their signature. Or, we rethink the
meaning of DataTable and provide more support for matrices.

On Wed, Apr 4, 2018 at 11:58 AM, Elizabeth Purdom
<epurdom at stat.berkeley.edu> wrote:
> Hello,
>
> I am trying to do a rbind a normal (in memory) matrix with a HDF5Matrix object or DelayedArray object and I am hitting problems. I’m using the development version of R and bioconductor (as of 2 weeks ago) and HDF5Array_1.7.9,   DelayedArray_0.5.23 — see sessionInfo at end of email.
>
> Basically if I apply rbind between my normal matrix and a HDF5Matrix, I get an error that a method doesn't exist for a DataTable class. If I force my HDF5Matrix object into a DelayedMatrix object using ‘as', I either get 1) a warning that it doesn’t know how to select a method (even if I use DelayedArray::rbind in my command) or 2) In addition to the warning, an error that the column names don’t match, even though neither object has colnames (they are NULL). Which of these I get depends on the order of the entries to rbind — the warning-only version occurs if the DelayedMatrix object is first.
>
> The warning-only option version gives the correct answer, but I want to understand how to avoid the warning, since I am using this in a package.
>
> It also seems like a bug that it would matter the order of the arguments, nor why I need to manually manipulate the HDF5Matrix into a DelayedMatrix object in order to do rbind.
>
> Thanks,
> Elizabeth Purdom
>
> Here is my code setting up the objects:
>
>> testdata<-matrix(rnorm(1000),nrow=100,ncol=10)
>> testhdf<-HDF5Array::writeHDF5Array(testdata, "./trash.h5")
>> class(testhdf)
> [1] "HDF5Matrix"
> attr(,"package")
> [1] “HDF5Array"
>
> Here are the errors/warnings I’m getting, some of which depend on the order of the entries into rbind:
>
>> test1<-rbind(testhdf,testdata) #error is independent of order of entries
> Error in rbind(...) :
>   missing 'rbind' method for DataTable class HDF5Matrix
>> test2<-DelayedArray::rbind(as(testhdf,"DelayedMatrix"),testdata)
> Warning message:
> In methods:::.selectDotsMethod(classes, .MTable, .AllMTable) :
>   multiple direct matches: "DelayedMatrix", "DataFrame"; using the first of these
>> test3<-DelayedArray::rbind(testdata,as(testhdf,"DelayedMatrix"))
> Error in rbind(...) :
>   column names for arg 2 do not match those of first arg
> In addition: Warning message:
> In methods:::.selectDotsMethod(classes, .MTable, .AllMTable) :
>   multiple direct matches: "DataFrame", "DelayedMatrix"; using the first of these
>
>> colnames(testdata)
> NULL
>> colnames(as(testhdf,"DelayedMatrix"))
> NULL
>
> Here is my sessionInfo:
>
>> sessionInfo()
> R Under development (unstable) (2018-03-22 r74446)
> Platform: x86_64-apple-darwin15.6.0 (64-bit)
> Running under: OS X El Capitan 10.11.6
>
> Matrix products: default
> BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
> LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
>
> locale:
> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
> other attached packages:
> [1] HDF5Array_1.7.9     rhdf5_2.23.5        DelayedArray_0.5.23 BiocParallel_1.13.3 IRanges_2.13.28
> [6] S4Vectors_0.17.38   BiocGenerics_0.25.3 matrixStats_0.53.1
>
> loaded via a namespace (and not attached):
> [1] compiler_3.5.0 tools_3.5.0    Rhdf5lib_1.1.5
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel