[R] Query about memory used in list and dataframe

[jim holtman]

I think you really have to show use your exact code that you did along
with an 'str' of each intermediate data structure since my quick test
does not bear out what you were saying:

> test.df <- data.frame(a1= 1:6, a2= 1:6, a3 = 1:6, a4 = 1:6, a5 = 1:6, a6 = 1:6
+     , a7=1:6, a8 = 1:6, a9 = 1:6, a10 = 1:6, a11 = 1:6, a12 = 1:6)
> test.df
  a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11 a12
1  1  1  1  1  1  1  1  1  1   1   1   1
2  2  2  2  2  2  2  2  2  2   2   2   2
3  3  3  3  3  3  3  3  3  3   3   3   3
4  4  4  4  4  4  4  4  4  4   4   4   4
5  5  5  5  5  5  5  5  5  5   5   5   5
6  6  6  6  6  6  6  6  6  6   6   6   6
> object.size(test.df)
2264 bytes
> x <- 1:6
> test.df1 <- data.frame(x,x,x,x,x,x,x,x,x,x,x,x)
> object.size(test.df1)
2264 bytes
> list.1 <- list(x,x,x,x,x,x,x,x,x,x,x,x)
> object.size(list.1)
1032 bytes
> list.2 <- NULL
> for (i in 1:12) list.2[[i]] <- x
> object.size(list.2)
1032 bytes
> test.df3 <- as.data.frame(do.call(cbind, list.2))
> object.size(test.df3)
2264 bytes
>
>
>


On Sat, May 5, 2012 at 5:50 PM, Shivam <shivamsingh at gmail.com> wrote:
> Hi,
>
> I had a query regarding which object, a list or a dataframe, consumes more
> R memory. Let me clarify this:
>
> For example, I have a df of 6 rows and 12 columns, say 'test'. I do
> object.size() and find it uses 3.3 KB of memory.
>
> I run a loop and make a list, say 'testlist', of 6 elements, each element
> being the above mentioned df 'test'. The size of this list is 19.9 KB,
> understandably.
>
> Now I combine this list into a dataframe using rbind. The df formed has 12
> cols and 36 rows. The size of this df is just 5.8 KB, almost a 75%
> reduction in memory.
>
> I had to work with a much larger list, and I thought of using the same
> method to convert my bigger list (62 dataframes, each having 4 cols and
> close to 200,000 rows) into a single dataframe. The big list, sat LIST A,
> had a size of 571 MB. But when I convert it into a dataframe, say DF A,
> using rbind, the object size increases to 1.35 GB. This was in
> contradiction to the earlier result.
>
> What am I missing? Why a 75% reduction in size in one case and double size
> in other? Anyone with any explanation?
>
> Sorry for the verbose email, just wanted to make my case clear.
>
> Thanks in advance,
> Regards
> Shivam
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



-- 
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.