Hi all,
 
I am currently building an application based on R 1.7.1 (+ compiled
C/C++ code + MySql + VB).  I am building this application to work on 2
different platforms (Windows XP Professional (500mb memory) and Windows
NT 4.0 with service pack 6 (1gb memory)).  This is a very memory
intensive application performing sophisticated operations on "large"
matrices (typically 5000x1500 matrices).
 
I have run into some issues regarding the way R handles its memory,
especially on NT.  In particular, R does not seem able to recollect some
of the memory used following the creation and manipulation of large data
objects.  For example, I have a function which receives a (large)
numeric matrix, matches against more data (maybe imported from MySql)
and returns a large list structure for further analysis.  A typical call
may look like this .
 
> myInputData <- matrix(sample(1:100, 7500000, T), nrow=5000)
> myPortfolio <- createPortfolio(myInputData)
 
It seems I can only repeat this code process 2/3 times before I have to
restart R (to get the memory back).  I use the same object names
(myInputData and myPortfolio) each time, so I am not create more large
objects ..
 
I think the problems I have are illustrated with the following example
from a small R session .
 
> # Memory usage for Rui process = 19,800
> testData <- matrix(rnorm(10000000), 1000) # Create big matrix
> # Memory usage for Rgui process = 254,550k
> rm(testData)
> # Memory usage for Rgui process = 254,550k
> gc()
         used (Mb) gc trigger  (Mb)
Ncells 369277  9.9     667722  17.9
Vcells  87650  0.7   24286664 185.3
> # Memory usage for Rgui process = 20,200k
 
In the above code, R cannot recollect all memory used, so the memory
usage increases from 19.8k to 20.2.  However, the following example is
more typical of the environments I use .
 
> # Memory 128,100k
> myTestData <- matrix(rnorm(10000000), 1000)
> # Memory 357,272k
> rm(myTestData)
> # Memory 357,272k
> gc()
          used (Mb) gc trigger  (Mb)
Ncells  478197 12.8     818163  21.9
Vcells 9309525 71.1   31670210 241.7
> # Memory 279,152k
 
Here, the memory usage increases from 128.1k to 279.1k
 
Could anyone point out what I could do to rectify this (if anything), or
generally what strategy I could take to improve this?
 
Many thanks,
Rich.
 
Mango Solutions
Tel : (01628) 418134
Mob : (07967) 808091
 

	[[alternative HTML version deleted]]


