[Bioc-devel] How can I maintain a stable list of source urls

[Tim Sears]

Thanks Mike.
On your last question I don't have a ready example but I noticed several
times that I had to update the sha256 hash but not the version number.
Possible I'm mistaken, so if I notice it again I'll post in this thread.

What is true for certain is that the nix R packages use source urls that
only get updated when someone runs r-modules generate-R-packages by hand
and updates the version numbers and sha256 hashes stored in
bioc-packages.nix or cran-packages.nix, part of the nixpkgs repo.  When you
install a package the source url is based on the last time these items were
checked by nixpkgs, and the same urls are queried to get the source. Even
if things worked one day, the next day the url is not guaranteed to exist
(or perhaps have the same sha256 as before). This makes it hard to use
nixpkgs to keep a development environment up to date.

I am working on some changes to nixpkgs to help with keeping nixpkgs more
up-to-date, but wasn't expecting to build and host custom mirrors as part
of the scope. Hope I can find a different solution.
Tim


On Fri, Apr 3, 2020 at 4:49 AM Mike Smith <grimbough using gmail.com> wrote:

> I was talking with another nix user a while back about this problem, and I
> don't think there's a good solution at the moment.
>
> One thing that strikes me is that when release packages get updated the
> old versions get put in an archive and the source tarball remains available
> e.g.
> https://bioconductor.org/packages/3.10/bioc/src/contrib/Archive/biomaRt/ Currently
> this archive is only created if a package is updated during the release
> cycle.  Would it be sufficient for all packages to get an archive URL as
> default?  Then if they aren't updated during a release cycle that URL is
> fine, and if there are updates you get a new link to the new package source.
>
> Something I'm not clear about is when you say this could be affected by
> "changes inside the repo that affect the hash output".  AFAIK a new tarball
> will only propagate to the download page if the version number has been
> bumped, and so a new link will be generated.  I'm not sure how it's
> possible to get a tarball with the same name but a different hash.
>
> On Tue, 31 Mar 2020 at 23:03, Tim Sears <tim using timsears.com> wrote:
>
>> Hi
>> I am working to make bioconductor packages available through the nix
>> package manager (https://nixos.org/). Currently nixpkgs points to a
>> source
>> url on one of the bioconductor mirrors. nixpkgs tracks the url and a
>> sha256
>> hash of the unpacked source link. Unfortunately the links themselves often
>> disappear as the packages get updated either due to a version number
>> change
>> or changes inside the repo that affect the hash output. This breaks
>> corresponding nix package. This happens even for stable bioconductor
>> versions like 3.10. nixpkgs aims for reproducible builds, which means when
>> things change, the url should reflect the change too and the contents
>> behich old urls should be preserved long enough to manage the  change. Do
>> you have any suggestions short of rebuilding a bioconductor mirror with
>> immutable urls?
>>
>> Thanks,
>> TIm
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioc-devel using r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>

	[[alternative HTML version deleted]]