[R] help usin scan on large matrix (caveats to what has been discussed before)

[peter dalgaard]

On Aug 12, 2010, at 1:34 PM, Martin Tomko wrote:

> Hi Peter,
> thank you for your reply. I still cannot get it to work.
> I have modified your code as follows:
> rows<-length(R)
> cols <- max(unlist(lapply(R,function(x) length(unlist(gregexpr(" ",x,fixed=TRUE,useBytes=TRUE))))))

Notice that the above is completely useless to the reader unless you tell us what R is (except for a statistical programming language ;-)) 

> c<-scan(file=f,what=rep(c(list(NULL),rep(list(0L),cols-1),rows-1)), skip=1)

What's the outer rep() and rows-1 doing in there???! Notice that the parentheses don't match up as I think you think they do, so there's really only one argument to rep(), making it a no-op. The rows-1 is going inside the c, which might be causing the apparent extra column. And the number of rows should not affect 'what=' anyway. Now if you had done what I wrote...

> m<-matrix(c, nrow = rows-1, ncol=cols+1,byrow=TRUE);

If you make a matrix from a list, odd things will happen. You need an unlist(c). And more than likely NOT byrow=TRUE. However, I think do.call(cbind,c) should do the trick more easily. 

> 
> the list c seems ok, with all the values I would expect. Still, length(c) gives me a value = cols+1, which I find odd (I would expect =cols).
> I thine repeated it rows-1 times (to account for the header row). The values seem ok.
> Anyway, I tried to construct the matrix, but when I print it, the values are odd:
> > m[1:10,1:10]
>      [,1] [,2]       [,3]       [,4]       [,5]       [,6]       [,7]
> [1,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [2,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [3,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [4,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [5,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [6,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [7,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [8,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [9,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> [10,] NULL Integer,15 Integer,15 Integer,15 Integer,15 Integer,15 Integer,15
> ....
> 
> Any idea where the values are gone?
> Thanks
> Martin
> 
> Hence, I filled it into the matrix of dimensions
> 
> On 8/12/2010 12:24 PM, peter dalgaard wrote:
>> On Aug 12, 2010, at 11:30 AM, Martin Tomko wrote:
>> 
>>   
>>> c<-scan(file=f,what=list(c("",(rep(integer(0),cols)))), skip=1)
>>> m<-matrix(c, nrow = rows, ncol=cols,byrow=TRUE);
>>> 
>>> for some reason I end up with a character matrix, which I don't want. Is this the proper way to skip the first column (this is not documented anywhere - how does one skip the first column in scan???). is my way of specifying "integer(0)" correct?
>>>     
>> No. Well, integer(0) is just superfluous where 0L would do, since scan only looks at the types not the contents, but more importantly, what= wants a list of as many elements as there are columns and you gave it
>> 
>>   
>>> list(c("",(rep(integer(0),5))))
>>>     
>> [[1]]
>> [1] ""
>> 
>> I think what you actually meant was
>> 
>> c(list(NULL),rep(list(0L),5))
>> 
>> 
>> 
>>   
>>> And finally - would any sparse matrix package be more appropriate, and can I use a sparse matrix for the image() function producing typical heat,aps? I have seen that some sparse matrix packages produce different looking outputs, which would not be appropriate.
>>> 
>>> Thanks
>>> Martin
>>> 
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>     
>>   
> 
> 
> -- 
> Martin Tomko
> Postdoctoral Research Assistant
> 
> Geographic Information Systems Division
> Department of Geography
> University of Zurich - Irchel
> Winterthurerstr. 190
> CH-8057 Zurich, Switzerland
> 
> email: 	martin.tomko at geo.uzh.ch
> site:	http://www.geo.uzh.ch/~mtomko
> mob: 	+41-788 629 558
> tel: 	+41-44-6355256
> fax: 	+41-44-6356848
> 

-- 
Peter Dalgaard
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com