This made my day. You're looking for R 3.1:

> mm <- matrix(as.integer(NA),1e7,20)
> print(object.size(mm), unit="Mb")
762.9 Mb
> gc()
            used  (Mb) gc trigger  (Mb)  max used  (Mb)
Ncells    799374  42.7    1265230  67.6    984024  52.6
Vcells 100903855 769.9  111583218 851.4 100907462 769.9
> df <- as.data.frame(mm)
> print(object.size(df), unit="Mb")
762.9 Mb
> gc()
            used   (Mb) gc trigger   (Mb)  max used   (Mb)
Ncells    799762   42.8    1265230   67.6    984024   52.6
Vcells 200904509 1532.8  231527856 1766.5 215905175 1647.3
> DF <- as(mm, "DataFrame")
> print(object.size(DF), unit="Mb")
762.9 Mb
> gc()
            used   (Mb) gc trigger   (Mb)  max used   (Mb)
Ncells    800938   42.8    1265230   67.6    984024   52.6
Vcells 300906552 2295.8  342980844 2616.8 310905457 2372.1

So at the end we are (rightfully) consuming 2295.8 MB, having only consumed
2372.1 MB during the operation.

Thanks,
Michael


On Tue, Jun 17, 2014 at 4:42 AM, Ludo Pagie <ludo.pagie@gmail.com> wrote:

> Dear all,
>
> I know I'm on slippery slopes here with stating some R-feature uses too
> much memory. Still I find something odd which hampers me in getting my job
> done. I make a DataFrame from a large integer matrix (rowaxcols = 7e6 x
> 300) and the process of creating the DataFrame consumes more memory than I
> have. I'm working on a Ubuntu machine.
>
> My question is: Am I overlooking something; can I change my code such that
> memory overhead is more reasonable. Or is there a problem with the
> implementation of DataFrame's, or is there another issue?
>
> Thanks for helping me out. Ludo
>
> (smaller) USE CASE:
> ##################
>
> # load IRanges
> library(IRanges)
>
> # create a largish matrix
> mm <- matrix(as.integer(NA),1e7,20)
> # consumes about 800 Mb
> print(object.size(mm), unit="Mb")
> # 762.9 Mb
>
> # at this point program top suggests that my R job consumes 900Mb (VIRT) ?
> 833Mb (RES) memory. That's reasonable I think
>
> # coerce the matrix into a regular data.frame
> df <- as.data.frame(mm)
> # also consumes about 800Mb
> print(object.size(df), unit="Mb")
> # 762.9 Mb
>
> # at this point program top suggests that my R job consumes 1800Mb (VIRT) ?
> 1750Mb (RES) memory. That's reasonable I think; about double the size as
> before.
>
> # corerce the same matrix into a DataFrame
> DF <- as(mm, "DataFrame")
> # also consumes about 800Mb
> print(object.size(DF), unit="Mb")
> # 762.9 Mb
>
> # but now top says that my R job takes 5500Mb (VIRT) / 5400Mb (RES)
> memory!!! That's 3700Mb for coercing a 800Mb object
>
> # sessioninfo
> > sessionInfo()
> R version 3.0.2 (2013-09-25)
> Platform: x86_64-unknown-linux-gnu (64-bit)
>
> locale:
>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>  [3] LC_TIME=nl_NL.UTF-8        LC_COLLATE=en_US.UTF-8
>  [5] LC_MONETARY=nl_NL.UTF-8    LC_MESSAGES=en_US.UTF-8
>  [7] LC_PAPER=nl_NL.UTF-8       LC_NAME=C
>  [9] LC_ADDRESS=C               LC_TELEPHONE=C
> [11] LC_MEASUREMENT=nl_NL.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] parallel  stats     graphics  grDevices utils     datasets  methods
> [8] base
>
> other attached packages:
> [1] IRanges_1.20.6     BiocGenerics_0.8.0
>
> loaded via a namespace (and not attached):
> [1] stats4_3.0.2
> ########################################################
>
>
> ---
> Ludo Pagie
> Netherlands Cancer Institute
> Gene Regulation (B4)
> van Steensel Group
> Plesmanlaan 121
> 1066 CX Amsterdam
> The Netherlands
>
> Tel.: ++ 20 512 7986
> Fax:  ++ 20 669 1383
> email: l.pagie@nki.nl
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioconductor mailing list
> Bioconductor@r-project.org
> https://stat.ethz.ch/mailman/listinfo/bioconductor
> Search the archives:
> http://news.gmane.org/gmane.science.biology.informatics.conductor
>

	[[alternative HTML version deleted]]

