[Rd] Package compression benchmarks for zstd vs gzip

[Ivan Krylov]

On Sat, 11 Jan 2025 16:05:46 -0800
Henrik Bengtsson <henrik.bengtsson using gmail.com> wrote:

> It's probably also worth looking at package compression with 'xz'
> compression. In [1], Mike FC has a graph where 'bzip2' and 'xz' seem
> to give the best compression ratios, at least for RDS files.

'bzip2' can be surprisingly good on very repetitive payloads. It
compresses 0x80000000 zero bytes to only 1.5 KiB, much better than 'xz
-9' with 305 KiB (with compression settings not making much
difference), although the compression is not perfect. One terabyte of
zeros can be compressed to 697202 bytes of repetitive compressed stream
that can be bzipped further to 248 bytes.

Binary packages are probably the most obvious target for new
compression methods because there is no need to install them on older
versions of R.

-- 
Best regards,
Ivan