[R] sparseM and kronecker product_R latest version

Mon Jan 11 16:20:57 CET 2010

Many thanks for it.

However it is strange that when I put the numbers rather than ncol(R)
(a matrix with ncol=36698) it worked. Look below

> dim(res2)
[1] 170471  25822
> D<- as.matrix.csr(0,nrow(tmpb),25822)
> D<- as.matrix.csr(0,nrow(tmpb),ncol(res2))
Error in if (length(x) == nrow * ncol) x <- matrix(x, nrow, ncol) else { :
  missing value where TRUE/FALSE needed
In addition: Warning message:
In nrow * ncol : NAs produced by integer overflow

But probably it is true what you said, anyway.
So, do you suggest me to use directly the simple matrix command? or a
kind of sparse matrix within your package?!? Many thanks again.

Moreover unfortunately I then run the procedure of quantile fixed
effects elaborated by Koenker on a sample of 276000 observations, and
even if I am  using a 64 bit machine with 6 gigabyte of Ram (and we
will raise it to 16 giga) he gives me the error :

Error: cannot allocate vector of size 155.4 Mb

and R cannot read more than 4 giga of RAM (even if I have 6).
Any suggestion to overcome this problem? Otherwise I have to randomly
subsample my data, which is a pity when you have such an amount of
information.

2010/1/11 Martin Maechler <maechler at stat.math.ethz.ch>:
>>>>>> "am" == alessia matano <alexis.rtd at gmail.com>
>>>>>>     on Mon, 11 Jan 2010 15:38:39 +0100 writes:
>
>    am> Many thanks for your suggestions,
>    am> fortunately when I closed R and opeedn it again, it did not give again
>    am> that error. However, now there is something more strange happening,
>    am> related again to sparse matrix, and I am afraid it could concern
>    am> memory problems (I put 4000 of memory limit).
>
>    am> I am defining a matrix where I have nrow=113289, ncol=36698 such as:
>
>    am> D<- as.matrix.csr(0, nrow, ncol)
>
>    am> Error in if (length(x) == nrow * ncol) x <- matrix(x, nrow, ncol) else { :
>    am> missing value where TRUE/FALSE needed
>    am> In addition: Warning message:
>    am> In nrow * ncol : NAs produced by integer overflow
>
>    am> With reduced number of rows and columns it worked, so I am wondering
>    am> wether it is a problem of memory.
>
> It's not; rather a "trap" that the programmers of SparseM fell
> into (I did too, in the past):  Your  nrow and ncol  are so
> large that  nrow * ncol is larger than R's maximal integer,
> ( == .Machine$integer.max  which is   2^31 - 1  on all current versions of R)
> and so overflows to NA and such leads to the
> "missing value" warning you see.
>
>    am> My pc has 6gb of nmemory, and it is
>    am> 64-bit windows system, but R reads only 4 has capacity.
>
>    am> Thanks for your help
>    am> alessia
>
> As a side note:
> As co-author of R package 'Matrix' in to which we have put a
> large amount of work and which is nowadays a recommended package
> (i.e., also part of every R distribution),
> I do wonder why you don't use 'Matrix' for sparse matrix
> computing in R.
>
> Best regards,
> Martin Maechler, ETH Zurich
>
>
>    am> 2010/1/11 David Winsemius <dwinsemius at comcast.net>:
>    >>
>    >> On Jan 11, 2010, at 7:55 AM, Peter Ehlers wrote:
>    >>
>    >>> Do you have the same problem with the example
>    >>> on the help page?
>    >>>
>    >>> ?'%x%-methods'
>    >>>
>    >>> Works for me on Windows Vista (32-bit OS) and
>    >>> R version 2.10.1 Patched (2010-01-05 r50896).
>    >>>
>    >>> -Peter Ehlers
>    >>>
>    >>> alessia matano wrote:
>    >>>>
>    >>>> Dear all,
>    >>>> I just installed the new version of R, 2.10.1, and I am currently
>    >>>> using the package sparseM. (I also use a 64 bit windows version)
>    >>
>    >> SparseM
>    >>
>    >>>> I got a problem that I never had: when I try to multiply with a
>    >>>> kronecker product (%x%) two sparse matrixes I get the following
>    >>>> message:
>    >>>> Error in dim(x) <- length(x) : invalid first argument
>    >>>> I never had this problem with previous versions of R.
>    >>
>    >> I get the same error as you do when trying the example on the cited help
>    >> page using SparseM 0.83 in a 64 bit Mac version of 2.10.1
>    >>
>    >>> A.csr %x% matrix(1:4,2,2)
>    >> Error in dim(x) <- length(x) : invalid first argument
>    >>
>    >>> sessionInfo()
>    >> R version 2.10.1 RC (2009-12-09 r50695)
>    >> x86_64-apple-darwin9.8.0
>    >>
>    >> locale:
>    >> [1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8
>    >>
>    >> attached base packages:
>    >> [1] grid      splines   stats     graphics  grDevices utils     datasets
>    >>  methods   base
>    >>
>    >> other attached packages:
>    >>  [1] SparseM_0.83       Matrix_0.999375-32 Epi_1.1.10         plotrix_2.7-2
>    >>      ROCR_1.0-4
>    >>  [6] gplots_2.7.4       caTools_1.10       bitops_1.0-4.1     gdata_2.6.1
>    >>      gtools_2.6.1
>    >> [11] lattice_0.17-26    Design_2.3-0       Hmisc_3.7-0
>    >>  survival_2.35-7
>    >>
>    >> loaded via a namespace (and not attached):
>    >> [1] cluster_1.12.1 tools_2.10.1
>    >>
>    >>
>    >>
>    >>>> May you help me?
>    >>>> thanks
>    >>>> alessia
>    >>>> ______________________________________________
>    >>>
>    >>> --
>    >>> Peter Ehlers
>    >>> University of Calgary
>    >>> 403.202.3921
>    >>>
>    >>> ______________________________________________
>    >>
>    >> David Winsemius, MD
>    >> Heritage Laboratories
>    >> West Hartford, CT
>    >>
>    >>
>
>    am> ______________________________________________
>    am> R-help at r-project.org mailing list
>    am> https://stat.ethz.ch/mailman/listinfo/r-help
>    am> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>    am> and provide commented, minimal, self-contained, reproducible code.
>