[Bioc-devel] Old package versions / Bioc archive of package's *.tar.gz releases?
Gabe Becker
becker.gabe at gene.com
Fri Oct 6 07:03:34 CEST 2017
Oh, and the svn/git walking knows about release branches so it will find
hot-patch versions correctly, as well.
~G
On Thu, Oct 5, 2017 at 10:01 PM, Gabe Becker <beckerg4 at gene.com> wrote:
> Correct. The actual order of checks is:
>
> 1. did switchr already retrieve that exact version for something else
> and keep it around,
> 2. A GRANRepository object if one is specified (don't worry much about
> this one)
> 3. the manifest itself (cran and bioc source types are ignored, but it
> would walk SCM if it had git/svn type manifest entry for the package)
> 4. cran repo and cran archives,
> 5. bioc repositories (all of them in descending order),
> 6. bioc git (bioc SVN for the version on CRAN, which appeared to still
> work last time I checked it a week or two ago)
>
> See the code in https://github.com/gmbecker/switchr/blob/master/
> R/retrievePkgVersion.R for details.
>
> Best,
> ~G
>
>
>
>
> On Oct 5, 2017 9:06 PM, "Henrik Bengtsson" <henrik.bengtsson at gmail.com>
> wrote:
>
> That's really nice; I didn't know it could do all that. For my
> clarification, when using PkgManifest(..., type = "bioc") it'll search
> (i) the CRAN archives, (ii) the Bioconductor repos(es), and then (iii)
> the Bioconductor Git repos - is that a correct observation? (I
> installed from https://github.com/gmbecker/switchr)
>
> /Henrik
>
>
>
>
> On Thu, Oct 5, 2017 at 4:15 PM, Gabe Becker <becker.gabe at gene.com> wrote:
> > In point of fact, it looks like IRanges 2.6.0 is an instance of that
> > weakness, so was probably a bad example. 2.6.1 installs correctly, or
> would
> > in it's native R/base bioc environment... (it fails for me in the
> library I
> > have...)
> >
> > Also, the version on CRAN uses the bioc SVN, so may not work for recent
> > versions.
> >
> > On Thu, Oct 5, 2017 at 3:58 PM, Gabe Becker <beckerg4 at gene.com> wrote:
> >>
> >> Henrik et al.,
> >>
> >> My switchr package (on CRAN, github at:
> >> http://github.com/gmbecker/switchr, preprint of the paper here:
> >> https://arxiv.org/abs/1501.02284) can do this.
> >>
> >> In fact, installing (cohorts of) old versions of packages is one of it's
> >> primary purposes. Specifically, it can install old source versions of
> >> packages from CRAN, Bioconductor, and general Git and SVN repos you
> tell it
> >> about.
> >>
> >> With the caveat that it's a bad idea in the general case to specify an
> old
> >> version of one package without specifying versions of its dependencies
> >> (switchr allows you to do this via a manifest, which can be constructed
> from
> >> sessionInfo output or guessed in the case of a CRAN package), you can
> just
> >> do
> >>
> >> > man = PkgManifest(name="IRanges", type="bioc")
> >>
> >> > install_packages("IRanges", man, versions = c(IRanges = "2.6.0"))
> >>
> >>
> >> And you will successfully completely break your Bioc installation by
> >> installing IRanges 2.6.0 into it. ;-)
> >>
> >> Switchr also gives you tools to more easilly maintain multiple libraries
> >> which contain, for example, different bioc versions in them.
> >>
> >> NB: switchr is subject to the caveat Martin pointed out and will fail to
> >> retrieve a buildable version of the package if said buildable version
> is not
> >> the first commit in SCM bearing that version in its DESCRIPTION file.
> >>
> >> Hope that helps.
> >>
> >> Best,
> >> ~G
> >>
> >> On Thu, Oct 5, 2017 at 2:21 PM, Martin Morgan
> >> <martin.morgan at roswellpark.org> wrote:
> >>>
> >>> On 10/05/2017 05:14 PM, Henrik Bengtsson wrote:
> >>>>
> >>>> On Thu, Oct 5, 2017 at 1:46 PM, Martin Morgan
> >>>> <martin.morgan at roswellpark.org> wrote:
> >>>>>
> >>>>> On 10/05/2017 01:50 PM, Henrik Bengtsson wrote:
> >>>>>>
> >>>>>>
> >>>>>> Is there an easily accessible archive for Bioconductor packages
> >>>>>> similar to what is provided on CRAN where you can find all released
> >>>>>> versions of a package, e.g.
> >>>>>> https://cran.r-project.org/src/contrib/Archive/PSCBS/?
> >>>>>>
> >>>>>> Say I want to access the source code for affy 1.18.0. Here are the
> >>>>>> two approaches I'm aware of and none of them are particularly
> >>>>>> appealing to me. Does anyone know of a better approach?
> >>>>>
> >>>>>
> >>>>>
> >>>>> The only option is to scrape, and that's approximate. One could build
> >>>>> an
> >>>>> archive
> >>>>>
> >>>>> pkg,version,branch,from_svn_rev,to_svn_rev
> >>>>>
> >>>>> and then consult that. Packages are supposed to increment the 'z' of
> >>>>> x.y.z,
> >>>>> but I'm sure there are many exceptions. I believe Jim Hester has an
> svn
> >>>>> script for this, but I wasn't able to locate it; it would be fast in
> >>>>> git.
> >>>>
> >>>>
> >>>> Thanks. About 'z' not being increased. Does the Bioc build servers
> >>>> release (a) continuously or (b) only when it detects a version change
> >>>> x.y.z -> x.y.z+1? If it does it continuously, then what x.y.z is
> >>>> installed does matter on when it was downloaded/installed, correct?
> >>>> On the other hand, if it only builds in when a version bump is
> >>>> detected, then one can at least narrow it down to a much narrow set of
> >>>> x.y.z submits (if multiple exists).
> >>>
> >>>
> >>> The builder only pushes for upward increments, so a commit without a
> >>> 'positive' version bump would be built but not pushed to the public
> >>> repository. I'm not sure how rigorously this policy was enforced
> before,
> >>> e.g., 2005.
> >>>
> >>> Of course there are exceptions, e.g., it is occasionally (at most one
> or
> >>> two times a release cycle) necessary to flush the public repository
> >>> entirely, and then whatever is built is pushed. And there is nothing
> >>> stopping the user from doing a check-out from svn. Perhaps others will
> chime
> >>> in with the gory / correct details.
> >>>
> >>> Martin
> >>>
> >>>
> >>>>
> >>>>>
> >>>>> For your future self, this
> >>>>>
> >>>>>
> >>>>> https://bioconductor.org/packages/3.5/bioc/src/contrib/Archi
> ve/S4Vectors/
> >>>>>
> >>>>> provides a hint of a change coming with the next release -- archives
> of
> >>>>> all
> >>>>> RELEASE package versions, starting in Bioc 3.6. (Kudos to Val for
> >>>>> implementing this)
> >>>>
> >>>>
> >>>> This is great! Thanks Val for this.
> >>>>
> >>>> Thanks
> >>>>
> >>>> Henrik
> >>>>
> >>>>>
> >>>>> Martin
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> # APPROACH 1: Download from http://bioconductor.org
> >>>>>>
> >>>>>> The best approach I know now is to try to guess the date when this
> was
> >>>>>> released in order to identify the Bioconductor release version.
> >>>>>> Something like this:
> >>>>>>
> >>>>>> 1. Guess around 2010.
> >>>>>>
> >>>>>> 2. Go to http://bioconductor.org/about/release-announcements/ and
> see
> >>>>>> what R versions were in use during 2010. I find R 2.6.x and R
> 2.7.x.
> >>>>>> The Bioc version for those R versions (same URL) are Bioc 2.1 and
> Bioc
> >>>>>> 2.2. Let's focus on Bioc 2.2 (because I happen to know that is the
> >>>>>> one)
> >>>>>>
> >>>>>> 3. Following the Bioc 2.2 link on above URL to get to
> >>>>>> http://bioconductor.org/packages/2.2/BiocViews.html.
> >>>>>>
> >>>>>> 4. Click through, one eventually gets to
> >>>>>> http://bioconductor.org/packages/2.2/bioc/html/affy.html
> >>>>>>
> >>>>>> 5. The "Source" link points to
> >>>>>>
> >>>>>> http://bioconductor.org/packages/2.2/bioc/src/contrib/affy_1
> .18.2.tar.gz
> >>>>>>
> >>>>>> Say I wanted affy 1.16.0 instead and I made the wrong guess in Step
> 2,
> >>>>>> I can extrapolate from (Bioc 2.2, affy 1.18.x) finding that I should
> >>>>>> go to Bioc 2.1 to find affy 1.16.x (because releases have even minor
> >>>>>> version numbers). It works, but is a bit tedious if you need to do
> >>>>>> this more than once.
> >>>>>>
> >>>>>> Also, I'm pretty sure I read somewhere that Bioc on archive the most
> >>>>>> recent package version under each release, which means there is no
> >>>>>> affy_1.18.0.tar.gz available for download. Is that correct?
> >>>>>>
> >>>>>>
> >>>>>> # APPROACH 2: Version control
> >>>>>>
> >>>>>> $ git clone https://git.bioconductor.org/packages/affy
> >>>>>> $ cd affy
> >>>>>>
> >>>>>> # Package releases/versions are not tagged
> >>>>>> $ git tag
> >>>>>> [empty]
> >>>>>>
> >>>>>> # Check Bioc release branches
> >>>>>> $ git branch -a
> >>>>>> * master
> >>>>>> remotes/origin/HEAD -> origin/master
> >>>>>> remotes/origin/RELEASE_1_0
> >>>>>> remotes/origin/RELEASE_1_0_branch
> >>>>>> remotes/origin/RELEASE_1_4
> >>>>>> remotes/origin/RELEASE_1_4_branch
> >>>>>> remotes/origin/RELEASE_1_5
> >>>>>> [...]
> >>>>>> remotes/origin/RELEASE_3_5
> >>>>>> remotes/origin/master
> >>>>>>
> >>>>>> That's back to above game of trying to narrow down which Bioc
> release
> >>>>>> I should look at. A similar approach is to look at the commit log:
> >>>>>>
> >>>>>> $ git log DESCRIPTION
> >>>>>>
> >>>>>> commit 35573048255b398f99ff1d3560906b2121912248
> >>>>>> Author: Herve Pages <hpages at fhcrc.org>
> >>>>>> Date: Mon Apr 24 19:50:57 2017 +0000
> >>>>>>
> >>>>>> bump x.y.z versions to odd y after creation of 3_5 branch
> >>>>>>
> >>>>>> git-svn-id:
> >>>>>>
> >>>>>>
> >>>>>> file:///home/git/hedgehog.fhcrc.org/bioconductor/trunk/madma
> n/Rpacks/affy at 129129
> >>>>>> bc3139a8-67e5-0310-9ffc-ced21a209358
> >>>>>>
> >>>>>> commit aa4c2d648658e8c2cca2baf651aea92df55a4392
> >>>>>> Author: Herve Pages <hpages at fhcrc.org>
> >>>>>> Date: Mon Apr 24 19:25:24 2017 +0000
> >>>>>>
> >>>>>> bump x.y.z versions to even y prior to creation of 3_5 branch
> >>>>>>
> >>>>>> git-svn-id:
> >>>>>>
> >>>>>>
> >>>>>> file:///home/git/hedgehog.fhcrc.org/bioconductor/trunk/madma
> n/Rpacks/affy at 129126
> >>>>>> bc3139a8-67e5-0310-9ffc-ced21a209358
> >>>>>>
> >>>>>> [...]
> >>>>>>
> >>>>>> and try to locate affy 1.18.0 by peeking at the DESCRIPTION file
> >>>>>> history.
> >>>>>>
> >>>>>> Does anyone know of a better/more automated approach?
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Henrik
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Bioc-devel at r-project.org mailing list
> >>>>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >>>>>>
> >>>>>
> >>>>>
> >>>>> This email message may contain legally privileged and/or confidential
> >>>>> information. If you are not the intended recipient(s), or the
> employee
> >>>>> or
> >>>>> agent responsible for the delivery of this message to the intended
> >>>>> recipient(s), you are hereby notified that any disclosure, copying,
> >>>>> distribution, or use of this email message is prohibited. If you
> have
> >>>>> received this message in error, please notify the sender immediately
> by
> >>>>> e-mail and delete this email message from your computer. Thank you.
> >>>
> >>>
> >>>
> >>> This email message may contain legally privileged
> and/or...{{dropped:2}}
> >>>
> >>>
> >>> _______________________________________________
> >>> Bioc-devel at r-project.org mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >>
> >>
> >>
> >>
> >> --
> >> Gabriel Becker, Ph.D
> >> Associate Scientist
> >> Bioinformatics and Computational Biology
> >> Genentech Research
> >
> >
> >
> >
> > --
> > Gabriel Becker, Ph.D
> > Associate Scientist
> > Bioinformatics and Computational Biology
> > Genentech Research
>
>
>
--
Gabriel Becker, Ph.D
Associate Scientist
Bioinformatics and Computational Biology
Genentech Research
[[alternative HTML version deleted]]
More information about the Bioc-devel
mailing list