[R] large data set, and RDBMS

Jean Eid jeaneid at chass.utoronto.ca
Wed Feb 23 14:43:36 CET 2005


I have this somewhat large data set that is given to me in a fixed width
format. The file itself is already 100MB (Maybe R can actually handle
this but I am trying to gain some experience in postgres and RODBC).  I
am using postgres to preprocess the file and connect to the database through the RODBC package.

My question is much before the processing of the database in R (Iknow this
is somewhat off topic). I have more than 6000 variables in the data and
postgres is outputing and  error that it cannot handle more than
1600 columns in one table. Is there any Linux database managers that
actually do do this.; Of course I can break it into 4 or 5 tables but I am
wondering if there is a better solution.

System is  a Linux 2.6.8
psql (PostgreSQL) 7.4.7


Thank you,

Jean




More information about the R-help mailing list