Dear Nathaniel!

h5read is in fact designed to be able to call a user defined function
h5read.<myclass>, but it is not yet fully implemented, respectively tested.
I stalled this because of the complexity of this task. But maybe you and the
Bioc-devel list can help.

I can imaging the following scenario:
- h5write can write the attr(foo, "class") <- "myclass"  attribute to the HDF5 object
  This is already set up, one can invoke this by using write.attributes=TRUE as you mentioned

  h5write is a generic function and one can write its own h5write.<myclass> function.

- Before h5read reads the object, it tries to read the class-attribute and invokes h5read.<myclass>
   which is defined somewhere outside rhdf5.
     
The problems I came across are:
1.) Usually, the h5read.<mycall> is implemented in some package "mypackage".
      How do I know, which package it is, if the package is not yet loaded? 
      Do we have to store an additional "BioCpackage" attribute in the HDF5 object?
2.) What happens, if the package provider changes the class definition in the next BioC-release?
      Do we have to store a package version number as well?
3.) How shall we deal with R-attributes? 
      HDF5 attributes are not able to store all R-attibutes, because HDF5-attributes are restricted to 
      a maximum size, R-attibutes can be almost as large as you like. One way would be
      to store attributes in a group called /obj.ATTRIBUTES.
      E.g. assume you have an R-object foo with attribute names = c("A","B",…) of length 2^30
      and geneNames = c("ENSGA","ENSGB",…)
      Should h5write write the following:
      /foo :  an HDF5 object, e.g. an integer array
      /foo.ATTRIBUTES : a group
      /foo.ATTRIBUTES/names : a string vector
      /foo.ATTRIBUTES/geneNames : a string vector
      This definitely breaks, if someone wants to write a list that contains both
      elements "foo" and "foo.ATTRIBUTES". Is this acceptable?
4.) What is the best standard for storing S3/S4-objects in HDF5? 
      Assume there is an object foo class baa with slots a = "integer", b = "double" and c = "mysecondclass"
      Should h5write write the following:
      /foo :  a group with attributes class="baa", BioCpackage="baapackage"
      /foo/slots : a group
      /foo/slots/a :  integer
      /foo/slots/b :  double
      /foo/slots/c :  a group with attributes class="mysecondclass", BioCpackage="mysecondpackage"
      /foo/slots/c/slots
      and assume foo has additional attributes as above h5write would write in addition:
      /foo.ATTRIBUTES : a group
      /foo.ATTRIBUTES/names : a string vector
      /foo.ATTRIBUTES/geneNames : a string vector
     This standard would allow the definition of a function that reads S3/S4-objects of any kind
     and still allow the user to define its own function h5read.<myclass>.  

What do you think about this? I guess that is the direction that you have in mind. Any other
suggestions and comments are welcome.

Bernd



On 07.08.2014, at 02:49, Nathaniel Hayden <nhayden@fhcrc.org> wrote:

> When reading from an hdf5 file I would like to automatically call a function I define when datasets of an arbitrary type (see: 'class') are read from an hdf5 file. Since it looks like the existing infrastructure (courtesy of the 'callGeneric' parameter in h5read) in rhdf5 was made for this, I would like to avoid duplicating work. But I can't find an example of the h5read.<classname> functionality indicated in the callGeneric description in the h5read man page.
> 
> A simple example is if the type is integer, I want as.integer to be automatically called on the read-in object before it gets passed back. But I intend to extend this to other Bioconductor classes of arbitrary complexity.
> 
> Based on the documentation, it seems like either using attr(foo "class") <- "integer" (in conjunction with h5write(<...>, write.attributes=TRUE) or adding a 'class' attribute through the h5writeAttribute interface should be enough to trigger the h5read.integer function upon calling h5read. Neither seems to work. Note that I can pass read.attributes=TRUE and the attributes get assigned the object (for example, the object comes back with a "class" attribute), but that's not exactly what I'm after.
> 
> In looking at the R/h5read.R source code, it looks like the block where the h5read.<classname> call gets set up (around line 59) queries the "class" attribute of the read-in obj before the h5 object's attributes are actually read, so the 'cl' variable never seems to get set.
> 
> Here's an example where I would expect h5read.<classname> to be invoked, but it doesn't:
> 
> library(rhdf5)
> h5read.integer <- function(obj) { as.integer(obj) } ## h5read.<classname>
> debug(h5read.integer)
> exists(paste("h5read","integer",sep="."),mode="function")
> 
> h5fl <- tempfile(fileext=".h5")
> h5createFile(h5fl)
> ints <- 42L:33L
> attr(ints, "class") <- "integer"
> h5write(ints, h5fl, "foo", write.attributes=TRUE)
> H5close()
> 
> ## h5writeAttribute route
> ##fid <- H5Fopen(h5fl)
> ##did <- H5Dopen(fid, "foo")
> ##h5writeAttribute("integer", did, name="class")
> ##H5close()
> 
> ##res <- h5read(h5fl, "foo", read.attributes=FALSE)
> res <- h5read(h5fl, "foo", read.attributes=TRUE)
> 
> Running the external h5dump utility confirms that a "class" attribute is attached to the foo DATASET, which seems to match what the h5read man page prescribes. If I edit the source code to set the 'cl' variable to "integer" my h5read.integer function gets invoked, as expected.
> 
> Any help would be much appreciated. Thank you.


	[[alternative HTML version deleted]]

