[R] splitting and saving a large dataframe

[Jean.Coursol at math.u-psud.fr]

for( i in levels(mydata$myfactor) ) {
   d = mydata[mydata$myfactor==i,]
   write.table(d, paste('d', i, '.data', sep='') )
   }

Jean Coursol


Quoting Dimitri Szerman <dimitrijoe at ipea.gov.br>:

> Hi,
>
> I searched for this in the mailing list, but found no results.
>
> I have a large dataframe ( dim(mydata)= 1297059      16, object.size(mydata=
> 145280576) ) , and I want to perform some calculations which can be done by
> a factor's levels, say, mydata$myfactor. So what I want is to split this
> dataframe into nlevels(mydata$myfactor) = 80 levels. But I must do this
> efficiently, that is, I can't actually split the dataframe and then save the
> resulting 80 smaller ones, as I would run out of memory. I don't know how to
> do it though.
>
> So, can anyone give a hint on how to get to get one smaller dataframe, save
> it into a file, remove it from the workspace, get the next smaller
> dataframe, save it into another file, remoe it from the workspace, and so
> on?
>
> Thank you,
>
> Dimitri
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>