[Rd] CRAN mirror size mushrooming; consider archiving some?

[Uwe Ligges]

On 25.07.2011 19:47, Paul Johnson wrote:
> Hi, everybody
>
> I'm setting up a new CRAN mirror and filled up the disk space the
> server allotted me.  I asked for more, then filled that up.  Now the
> system administrators want me to buy an $800 fiber channel card and a
> storage device.  I'm going to do that, but it does make want to
> suggest to you that this is a problem.

Why? Just for the mirror? That's nonsense. A 6 year old outdated desktop 
machine (say upgraded to 2GB RAM) with a 1T harddisc for 50$ should be 
fine for your first tries. The bottleneck will probably be your network 
connection rather than the storage.


> CRAN now is about 68GB, and about 3/4 of that is in the bin folder,
> where one finds copies of compiled packages for macosx and windows.
> If the administrators of CRAN would move the packages for R before,
> say, 2.12, to long term storage, then mirror management would be a bit
> more, well, manageable.
>
> Moving the R for windows packages for, say, R 2.0 through 2.10 would
> save some space, and possibly establish a useful precedent for the
> long term.


That is right, but then users of R < 2.11.0 could no longer use 
install.packages() and friends. If we want to move stuff around in 
future, we may want to implement that in R first. We thought about 
removing old binaries before, but then disk space increased roughly as 
exponentially as repository space in the past and we decided to stay 
with it as is.


> Here's the bin/windows folder. Note it is expanding exponentially (or nearly so)

And you see that quite a lot of efforts were made during the last 
release cycles to reduce the amount of used memory (e.g. using better 
compression).

Best wishes,
Uwe



> $ du --max-depth=1 | sort
> 1012644 ./2.6
> 103504  ./1.7
> 122200  ./1.8
> 1239876 ./2.7
> 1487024 ./2.8
> 15220   ./ATLAS
> 167668  ./1.9
> 17921604        .
> 1866196 ./2.9
> 204392  ./2.0
> 2207708 ./2.10
> 2340120 ./2.13
> 2356272 ./2.12
> 2403176 ./2.11
> 298620  ./2.1
> 364292  ./2.2
> 438044  ./2.3
> 595920  ./2.4
> 698064  ./2.5
>