[R] Query about memory used in list and dataframe

[Rui Barradas]

Hello,

Also, note that to rbind data frames, like the op says, takes less memory
than to cbind.


x <- 1:6
list.1 <- list(x,x,x,x,x,x,x,x,x,x,x,x)
test.df3 <- as.data.frame(do.call(cbind, list.2))
test.df4 <- as.data.frame(do.call(rbind, list.2))

object.size(list.1)
object.size(test.df3)
object.size(test.df4)

(More columns == more variables == more memory.)

Rui Barradas


jholtman wrote
> 
> I think you really have to show use your exact code that you did along
> with an 'str' of each intermediate data structure since my quick test
> does not bear out what you were saying:
> 
>> test.df <- data.frame(a1= 1:6, a2= 1:6, a3 = 1:6, a4 = 1:6, a5 = 1:6, a6
>> = 1:6
> +     , a7=1:6, a8 = 1:6, a9 = 1:6, a10 = 1:6, a11 = 1:6, a12 = 1:6)
>> test.df
>   a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12
> 1  1  1  1  1  1  1  1  1  1   1   1   1
> 2  2  2  2  2  2  2  2  2  2   2   2   2
> 3  3  3  3  3  3  3  3  3  3   3   3   3
> 4  4  4  4  4  4  4  4  4  4   4   4   4
> 5  5  5  5  5  5  5  5  5  5   5   5   5
> 6  6  6  6  6  6  6  6  6  6   6   6   6
>> object.size(test.df)
> 2264 bytes
>> x <- 1:6
>> test.df1 <- data.frame(x,x,x,x,x,x,x,x,x,x,x,x)
>> object.size(test.df1)
> 2264 bytes
>> list.1 <- list(x,x,x,x,x,x,x,x,x,x,x,x)
>> object.size(list.1)
> 1032 bytes
>> list.2 <- NULL
>> for (i in 1:12) list.2[[i]] <- x
>> object.size(list.2)
> 1032 bytes
>> test.df3 <- as.data.frame(do.call(cbind, list.2))
>> object.size(test.df3)
> 2264 bytes
>>
>>
>>
> 
> 
> On Sat, May 5, 2012 at 5:50 PM, Shivam <shivamsingh@> wrote:
>> Hi,
>>
>> I had a query regarding which object, a list or a dataframe, consumes
>> more
>> R memory. Let me clarify this:
>>
>> For example, I have a df of 6 rows and 12 columns, say 'test'. I do
>> object.size() and find it uses 3.3 KB of memory.
>>
>> I run a loop and make a list, say 'testlist', of 6 elements, each element
>> being the above mentioned df 'test'. The size of this list is 19.9 KB,
>> understandably.
>>
>> Now I combine this list into a dataframe using rbind. The df formed has
>> 12
>> cols and 36 rows. The size of this df is just 5.8 KB, almost a 75%
>> reduction in memory.
>>
>> I had to work with a much larger list, and I thought of using the same
>> method to convert my bigger list (62 dataframes, each having 4 cols and
>> close to 200,000 rows) into a single dataframe. The big list, sat LIST A,
>> had a size of 571 MB. But when I convert it into a dataframe, say DF A,
>> using rbind, the object size increases to 1.35 GB. This was in
>> contradiction to the earlier result.
>>
>> What am I missing? Why a 75% reduction in size in one case and double
>> size
>> in other? Anyone with any explanation?
>>
>> Sorry for the verbose email, just wanted to make my case clear.
>>
>> Thanks in advance,
>> Regards
>> Shivam
>>
>>        [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-help@ mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
> 
> 
> 
> -- 
> Jim Holtman
> Data Munger Guru
> 
> What is the problem that you are trying to solve?
> Tell me what you want to do, not how you want to do it.
> 
> ______________________________________________
> R-help@ mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
> 


--
View this message in context: http://r.789695.n4.nabble.com/Query-about-memory-used-in-list-and-dataframe-tp4611896p4612876.html
Sent from the R help mailing list archive at Nabble.com.