[R] Computing very large distance matrix

[Prof Brian Ripley]

On Wed, 25 Feb 2004, Arnav Sheth wrote:

> 
> Hello All,
> 
> I have a 131072x132 matrix for which I need to compute a regular euclidean distance matrix, which I then need to transform and run agnes() on this transformed matrix. I am having trouble computing the distance matrix as it is fairly large and I am sure I have gone over the max. 
> 
> The specific error I am getting is:
> Error in vector("double", length) : negative length vectors are not allowed

We've improved that message for the next release.

> I have increased the memory limit to the maximum capacity of my hard
> drive (which is around 20gb), with no success.

You'd need to increase the 32-bit limit of Windows, too ....

> I am running the RGUI on Windows XP with 512 mb of RAM.
> 
> Would anyone have any suggestions as to how I can overcome this problem?
> I would be most grateful for any help.

No. The distance matrix is 8Gb all by itself.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595