[R] Matrix - querying dsCMatrix how to save it

[Martin Maechler]

I was asked privately

    > I have been using <the Matrix package> extensively to
    > calculate the relatedness-matrix based on the pedigree for
    > our ornamental plants.  I was wondering if you could give
    > me some advice, for which I would like to thank you in
    > advance!

    > The output of the relatedness matrix I get from
    > the nadiv package is a dsCMatrix. I would like to save it
    > in some way, so I could query a subset of this matrix
    > later on, as in most cases, the full matrix is not
    > needed. Do you know what kind of format is ideal for
    > data-storage (either in file or in a database) for a
    > sparse matrix?  Thanks in advance!

as that is a relative general question, I allow myself to answer
in public such that the answer my help others as well.

Let's use an example from package 'nadiv':

require(nadiv)

?makeA # to learn about the function

## create a largish example
Awarcol <- makeA(ggTutorial[1:2000, 1:3])

str(Awarcol)
nnzero(Awarcol)
object.size(Awarcol)
object.size(as.matrix(Awarcol)) # ~ 14 times large

image(Awarcol) # wait a bit .. ... but it's worth it!

-------------

> Awarcol <- makeA(ggTutorial[1:2000, 1:3])
> str(Awarcol)
Formal class 'dsCMatrix' [package "Matrix"] with 7 slots
  ..@ i       : int [1:164107] 0 1 2 3 4 5 6 7 8 9 ...
  ..@ p       : int [1:2001] 0 1 2 3 4 5 6 7 8 9 ...
  ..@ Dim     : int [1:2] 2000 2000
  ..@ Dimnames:List of 2
  .. ..$ : chr [1:2000] "1" "2" "3" "4" ...
  .. ..$ : chr [1:2000] "1" "2" "3" "4" ...
  ..@ x       : num [1:164107] 1 1 1 1 1 1 1 1 1 1 ...
  ..@ uplo    : chr "U"
  ..@ factors : list()

> nnzero(Awarcol)
[1] 326214
> object.size(Awarcol)
2235112 bytes
> object.size(as.matrix(Awarcol))
32256488 bytes
>
------------------------------------------------

Now, save it (as file to current working directory) via

saveRDS(Awarcol, file = "makeA_warc.rds")

## and that is *only* 110 Kbytes :
> file.size("makeA_warc.rds")
[1] 115288


Or restore it in a later R session

Amat.warcol <- readRDS("makeA_warc.rds")

> str(Awarc)
Loading required package: Matrix
Formal class 'dsCMatrix' [package "Matrix"] with 7 slots
  ..@ i       : int [1:164107] 0 1 2 3 4 5 6 7 8 9 ...
  ..@ p       : int [1:2001] 0 1 2 3 4 5 6 7 8 9 ...
  ..@ Dim     : int [1:2] 2000 2000
  ..@ Dimnames:List of 2
  .. ..$ : chr [1:2000] "1" "2" "3" "4" ...
  .. ..$ : chr [1:2000] "1" "2" "3" "4" ...
  ..@ x       : num [1:164107] 1 1 1 1 1 1 1 1 1 1 ...
  ..@ uplo    : chr "U"
  ..@ factors : list()
> image(Awarc)
>

Note how 'str(..)' automatically loaded the Matrix package
which you do need for  image(<dsCMatrix>) to work nicely.


Best regards,
Martin

--
Martin Maechler
ETH Zurich  and  R Core team