[R-sig-Debian] pinning of binary r-cran-* packages from c2d4u / r2u on Ubuntu 22.04

[Thomas Petzoldt]

Am 09.08.2023 um 18:17 schrieb Dirk Eddelbuettel:
> 
> Hi Thomas,
> 
> Thanks a lot for bringing this over from r-package-devel!

And thanks for the quick help from my side!

> As for your question, "there is a lot of meat on these bones" and I really
> think we should discuss this here.
> 
> Quick comments inline, more recap at the bottom.
> 
> On 9 August 2023 at 17:29, Thomas Petzoldt wrote:
> | Hi,
> |
> | I am running a couple of shiny servers with several apps that are based
> | around own CRAN packages. It worked stable for years, but due to the
> | growing number of packages, the compile time for regular manual package
> | installation and updates became inconvenient.
> |
> | Therefore, I have been very happy to use pre-compiled packages from the
> | c2d4u repository  -- and from now on testing r2u following a hint from
> | Dirk Edelbuettel. @Dirk: installation worked like a charm.
> 
> Glad to hear!

Indeed. Just updated 58 packages and all apps continued working.


> It also generally works flawlessly (and repeatedly) for myself and others in
> many context involving many packages.  Per the logs, we have shipped over 6.5
> million packages in the ~ 15 months it has been up and generally do more than
> 10k each day now.
> 
> | The downside is, that the r-cran-* binaries are being installed
> | automatically, together with the system update. So experienced repeated
> | cases that crashed some of the shiny apps. The main reason with c2d4u
> | were conflicts between the binary packages and other packages installed
> | from sources. This may be less relevant with r2u, but in the interest of
> | reproducibility and stability, I would prefer conservative manual updates.
> 
> Yes. If and when that happens, it is a bug!

Not necessarily. Breaking code can also be due to an incompatible 
feature update. We know that R has both, conservative and less 
conservative universes of packages ;-) Security is another issue.

> The correct (but laborious way) is to file and report them.

Hmm, depends. I am happy to contribute bug reports, but need evidence 
that it's a bug and not a feature - or configuration issue on my side.

> I think the overall issue here is the mix and match from 'distro' packages,
> and added packages. Given the complete coverage, I prefer r2u for
> consistency.  Which is why it is pinned higher at 700.

Agreed, so I applied the r2u pinning as descibed on your page.
> But one can have issue if because of ("semi-random") package renmaing /
> versioning / use of epoch [a leading version digit overriding] the distro
> wins.
> 
> Eg we had a recent issue for r2u when following the Debian and the R 4.3.*
> catchup, several packages (all around the graphics API that needed a rebuilt)
> got force-rebuilt and ended up with, say, 1.2.3-2 beating 1.2.3-1.c2u2204.1.
> So I quickly rebuilt that dozen. It was more of an issue for jammy (22.04)
> than focal as the (distro) focal packages are generally way behind what CRAN
> and hence r2u has.
> 
> The only really reliable way to fix this is by reporting the bugs the old
> school way so that r2u can take care of it.  I would need help from users and
> that is why I am asking for it here.
>   
> | My question: what is best practise, to disallow automatic updates for
> | all (or part of) r-cran-* packages? Uncommenting the complete package
> | source in the apt/sources.list.d/cd4u...list file? Fiddling around with
> | /etc/apt/preferences ?
> |
> | The ideal approach would be to put a plain textfile of all installed
> | r-cran packages somewhere to the system, where packages that are to be
> | upgraded (or oppositely: pinned) are just commented or outcommented.
> 
> I mentioned in the initial short reply on the other list I can see several
> options (now up to four)
> 
> 1) use dpkg to put packages on 'hold'. They will not be touched or
> updated. Reliable, manual, tedious for many. See 'man dpkg'.

This (or same with apt-mark) is probably the easiest: Write a shell (or 
R) script iterating over a text file of packages. Should work similar to 
a traditional "install.packsges()"-script if almost all packages are 
from r2u and only a few directly from Github.

> 2) look into 'apt pinning' via preferences and its wiki page at
> https://wiki.debian.org/AptConfiguration etc   This may work, I have not had
> the patience to work the details out.

Same to me.

>  But there are other value beyond the
> '500 to 900' default range. This may work, and be quite elegant.
> 
> 3) as you hint, just 'hide' the sources.list for r2u or c2d4u but it still
> risks updates from the distro versions :-/  Maybe 'easiest yet worst'.

Good point. This is what I have been afraid of.

> 4) maybe go the other way use eg the Rocker Project container for shiny and
> run you sacred production in a shielded setting? This sounds like more work,
> but it may be better. If it works for you depends.

Yes, I' just discussed a switch to docker with my colleague during 
lunch. Maybe the university will install a central Shiny service in the 
future or we buy the service ... For now, I am still happy with my very 
simplistic solution with one VM running a load balancing Apache proxy 
and one or more other VMs with multiple shiny server instances (the free 
version). It has its limitations, but is a pragmatic setting for 
research projects and university courses.

> Let's keep at this and see if we can make something 'already pretty good'
> even better!

All right, I'll stay tuned and give feedback.
> 
> Cheers,  Dirk
> 

Thanks a lot,

Thomas