[R] help usin scan on large matrix (caveats to what has been discussed before)
peter dalgaard
pdalgd at gmail.com
Thu Aug 12 16:29:04 CEST 2010
On Aug 12, 2010, at 1:34 PM, Martin Tomko wrote:
> Hi Peter,
> thank you for your reply. I still cannot get it to work.
> I have modified your code as follows:
> rows<-length(R)
> cols <- max(unlist(lapply(R,function(x) length(unlist(gregexpr(" ",x,fixed=TRUE,useBytes=TRUE))))))
Notice that the above is completely useless to the reader unless you tell us what R is (except for a statistical programming language ;-))
> c<-scan(file=f,what=rep(c(list(NULL),rep(list(0L),cols-1),rows-1)), skip=1)
What's the outer rep() and rows-1 doing in there???! Notice that the parentheses don't match up as I think you think they do, so there's really only one argument to rep(), making it a no-op. The rows-1 is going inside the c, which might be causing the apparent extra column. And the number of rows should not affect 'what=' anyway. Now if you had done what I wrote...
> m<-matrix(c, nrow = rows-1, ncol=cols+1,byrow=TRUE);
If you make a matrix from a list, odd things will happen. You need an unlist(c). And more than likely NOT byrow=TRUE. However, I think do.call(cbind,c) should do the trick more easily.
>
> the list c seems ok, with all the values I would expect. Still, length(c) gives me a value = cols+1, which I find odd (I would expect =cols).
> I thine repeated it rows-1 times (to account for the header row). The values seem ok.
> Anyway, I tried to construct the matrix, but when I print it, the values are odd:
> > m[1:10,1:10]
> [,1] [,2] [,3] [,4] [,5] [,6] [,7]
> [1,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [2,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [3,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [4,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [5,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [6,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [7,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [8,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [9,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [10,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> ....
>
> Any idea where the values are gone?
> Thanks
> Martin
>
> Hence, I filled it into the matrix of dimensions
>
> On 8/12/2010 12:24 PM, peter dalgaard wrote:
>> On Aug 12, 2010, at 11:30 AM, Martin Tomko wrote:
>>
>>
>>> c<-scan(file=f,what=list(c("",(rep(integer(0),cols)))), skip=1)
>>> m<-matrix(c, nrow = rows, ncol=cols,byrow=TRUE);
>>>
>>> for some reason I end up with a character matrix, which I don't want. Is this the proper way to skip the first column (this is not documented anywhere - how does one skip the first column in scan???). is my way of specifying "integer(0)" correct?
>>>
>> No. Well, integer(0) is just superfluous where 0L would do, since scan only looks at the types not the contents, but more importantly, what= wants a list of as many elements as there are columns and you gave it
>>
>>
>>> list(c("",(rep(integer(0),5))))
>>>
>> [[1]]
>> [1] ""
>>
>> I think what you actually meant was
>>
>> c(list(NULL),rep(list(0L),5))
>>
>>
>>
>>
>>> And finally - would any sparse matrix package be more appropriate, and can I use a sparse matrix for the image() function producing typical heat,aps? I have seen that some sparse matrix packages produce different looking outputs, which would not be appropriate.
>>>
>>> Thanks
>>> Martin
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>
>
> --
> Martin Tomko
> Postdoctoral Research Assistant
>
> Geographic Information Systems Division
> Department of Geography
> University of Zurich - Irchel
> Winterthurerstr. 190
> CH-8057 Zurich, Switzerland
>
> email: martin.tomko at geo.uzh.ch
> site: http://www.geo.uzh.ch/~mtomko
> mob: +41-788 629 558
> tel: +41-44-6355256
> fax: +41-44-6356848
>
--
Peter Dalgaard
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
More information about the R-help
mailing list