[Rd] CRAN mirror size mushrooming; consider archiving some?
ligges at statistik.tu-dortmund.de
Tue Jul 26 20:07:38 CEST 2011
On 25.07.2011 19:47, Paul Johnson wrote:
> Hi, everybody
> I'm setting up a new CRAN mirror and filled up the disk space the
> server allotted me. I asked for more, then filled that up. Now the
> system administrators want me to buy an $800 fiber channel card and a
> storage device. I'm going to do that, but it does make want to
> suggest to you that this is a problem.
Why? Just for the mirror? That's nonsense. A 6 year old outdated desktop
machine (say upgraded to 2GB RAM) with a 1T harddisc for 50$ should be
fine for your first tries. The bottleneck will probably be your network
connection rather than the storage.
> CRAN now is about 68GB, and about 3/4 of that is in the bin folder,
> where one finds copies of compiled packages for macosx and windows.
> If the administrators of CRAN would move the packages for R before,
> say, 2.12, to long term storage, then mirror management would be a bit
> more, well, manageable.
> Moving the R for windows packages for, say, R 2.0 through 2.10 would
> save some space, and possibly establish a useful precedent for the
> long term.
That is right, but then users of R < 2.11.0 could no longer use
install.packages() and friends. If we want to move stuff around in
future, we may want to implement that in R first. We thought about
removing old binaries before, but then disk space increased roughly as
exponentially as repository space in the past and we decided to stay
with it as is.
> Here's the bin/windows folder. Note it is expanding exponentially (or nearly so)
And you see that quite a lot of efforts were made during the last
release cycles to reduce the amount of used memory (e.g. using better
> $ du --max-depth=1 | sort
> 1012644 ./2.6
> 103504 ./1.7
> 122200 ./1.8
> 1239876 ./2.7
> 1487024 ./2.8
> 15220 ./ATLAS
> 167668 ./1.9
> 17921604 .
> 1866196 ./2.9
> 204392 ./2.0
> 2207708 ./2.10
> 2340120 ./2.13
> 2356272 ./2.12
> 2403176 ./2.11
> 298620 ./2.1
> 364292 ./2.2
> 438044 ./2.3
> 595920 ./2.4
> 698064 ./2.5
More information about the R-devel