[R] Allocate virtual memory on hard drive

[Jie]

Thank you for your advice.
Yes, I was wrong about the size. It seems that gc() does not return
the memory to the system, so my R process looks like taking more space
than yours.
R 3.0.0 solves the issue. Thanks.

Best wishes,
Jie

On Mon, Mar 11, 2013 at 1:05 PM, Prof Brian Ripley
<ripley at stats.ox.ac.uk> wrote:
> On 11/03/2013 16:45, Jie wrote:
>>
>> The vector contains 1.5*10^8 numeric elements. It takes about 3~4 GB in
>> memory.
>> And I would like to find percentiles: 0%, 0.5%, 1%, ... 100%
>> I use 64 bit R and windows 7 with 24GB Ram.
>
>
> So:
>
> 1) Try R 3.0.0 alpha.  Many operations on large vectors are more efficient
> there.
>
> 2) You could try --max-mem-size=32G, say.  In my experience Windows virtual
> memory management is too slow to be useful, but you could try ....
>
> 3) Add more RAM.  24GB is not a lot these days.
>
> However, I tried this on a Linux box. Such a vector is only just over 1GB
> and the maximum memory usage was 2.9GB.  Have you really told us the true
> story?
>
>
>> Thank you.
>>
>> Best,
>>
>>
>> On Mon, Mar 11, 2013 at 12:40 PM, jim holtman <jholtman at gmail.com> wrote:
>>>
>>> R runs with data in memory.  What type of system are you running on (32
>>> or
>>> 64 bit)?  How big is your data; you did not provide much information
>>> about
>>> your problem.  Depending on what you what to 'sort', there might be other
>>> ways of doing it.  This gets back to my tag line: "Tell me what you want
>>> to
>>> do, not how you want to do it".
>>>
>>> On Mon, Mar 11, 2013 at 11:20 AM, Jie <jimmycloud at gmail.com> wrote:
>>>>
>>>>
>>>> Dear All,
>>>>
>>>> I have a long sequence and want to find the quantile, or sort it first.
>>>> It seems sort() or quantile() reaches the memory limit.
>>>> Is there a way to allocate more memoy on SSD for R when startup, so
>>>> that R can use both RAM and hard drive space?
>>>> Thank you.
>>>>
>>>> Best wishes,
>>>> Jie
>>>>
>>>> ______________________________________________
>>>> R-help at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide
>>>> http://www.R-project.org/posting-guide.html
>>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Jim Holtman
>>> Data Munger Guru
>>>
>>> What is the problem that you are trying to solve?
>>> Tell me what you want to do, not how you want to do it.
>>
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>
> --
> Brian D. Ripley,                  ripley at stats.ox.ac.uk
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595