[R] Limit

[Val]

The data was read. The problem is with processing.

On Fri, Nov 8, 2024 at 7:30 PM Bert Gunter <bgunter.4567 using gmail.com> wrote:
>
> Is the problem reading the file in or processing it after it has been read in?
>
> Bert
>
> On Fri, Nov 8, 2024 at 5:13 PM Jeff Newmiller via R-help <r-help using r-project.org> wrote:
>>
>> Can you tell us what is wrong with the "chunked" package which comes up when you Google "r read large file in chunks"?
>>
>> On November 8, 2024 4:58:18 PM PST, Val <valkremk using gmail.com> wrote:
>> >Hi All,
>> >
>> >I am reading data file ( > 1B rows) and do some date formatting like
>> >      dat=fread(mydatafile)
>> >     dat$date1 <- as.Date(ymd(dat$date1))
>> >
>> >However, I am getting an error message saying that
>> >    Error: cons memory exhausted (limit reached?)
>> >
>> >The  script was working  when the number rows were  around 650M.
>> >
>> >Is there another way to  handle  a big data set in R?
>> >
>> >
>> >Thank you.
>> >
>> >______________________________________________
>> >R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> >https://stat.ethz.ch/mailman/listinfo/r-help
>> >PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
>> >and provide commented, minimal, self-contained, reproducible code.
>>
>> --
>> Sent from my phone. Please excuse my brevity.
>>
>> ______________________________________________
>> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide https://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.