[R] Importing big plain files from ERP-System/Data Mining with R

[Prof Brian Ripley]

On Tue, 26 Oct 2004 r-help.20.stefan817 at spamgourmet.com wrote:

> how can I import really big plain text data files (several GB) from an

Unlikely unless you have a 64-bit platform.

Only starting with R 2.0.0 can some 32-bit versions of R access files >
2Gb, and to import the file into R you need enough address space in R for
the object, which is normally more than the file size.

Almost certainly not if the unmentioned platform is Windows, but you could 
access the data from a DBMS.


> ERP-System (SAP-Tables) to R?
> The Header of these files are always similar, for example:
> 
> Tabelle:        T009
> Angezeigte Felder:  7 von  7  Feststehende Führungsspalten: 2  Listbreite
> 0250
> ----------------------------------------------------------------------
> |X|MANDT|PERIV|XKALE|XJABH|ANZBP|ANZSP|LTEXT                         |
> ----------------------------------------------------------------------
> |X|001  |01   |X    |     |012  |02   |ABC                           |
> |X|001  |V9   |     |     |012  |04   |Okt. - Sep., 4 Sonderperioden |
> |X|001  |WK   |     |X    |053  |00   |Kalenderwochen                |
> ----------------------------------------------------------------------
> 
> (including the first 5 rows in each downloaded table, row # 4 =field names,
> length of 1 row > 1023 bytes, count of fields > 256, size = several GB,
> count records = several million)
> 
> What is an appropriate way to read such tables in?
> 
> Greetings
> Stefan
> 
> P.S. I am a beginner with R. Until now I have used ACL (http://www.acl.com)
> for data mining purposes and I'm doing now my first try with R.
> Yes, I have
> [X] Read R Data Import/Export
> [X] Read Using R for Data Analysis
> [X] Read Simple R
> [X] Read Manuals
> [X] Read read.table() and scan() command

but you have not told us your platform.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595