[R] Importing Big data to R

[Peter Dalgaard]

> On 12 Jul 2017, at 08:44 , Mangalani Peter Makananisa <pmakananisa at sars.gov.za> wrote:
> 
> Dear R-Gurus,
> 
> I am trying to read in data with 24,349,113 rows to R-3.3.3 (64 bit) and have used  the library   "data.table"  and It managed to read 23,347,070 rows  and the remainder was 2,043 rows only.

I make that 1002043 rows... (typo?). One possibility is that you are skipping lines because of something like quote characters in data or some input lines being one field short. You might also simply be running out of memory.

However,

1) data.table is a contributed package. Its maintainer may be a better point of contact.

2) if you hope to get advice from the R community at large, it would be useful to tell us what you did, what kind of data you try to read using which command, etc. And, as Jim Holtman already said: error messages, if any. 

-pd

> 
> Could you please advise me as to which library/R-commands is suitable to read the full data in to R?
> 
> Kind regards,
> 
> Mangalani Peter Makananisa (5786)
> South African Revenue Service (SARS)
> +2782 456 4669 / +2712 422 7357
> 
> Please Note: This email and its contents are subject to our email legal notice which can be viewed at http://www.sars.gov.za/Pages/Email-disclaimer.aspx
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com