[Bioc-devel] Old package versions / Bioc archive of package's *.tar.gz releases?
Gabe Becker
becker.gabe at gene.com
Fri Oct 6 07:04:22 CEST 2017
On Thu, Oct 5, 2017 at 10:03 PM, Gabe Becker <beckerg4 at gene.com> wrote:
> Oh, and the svn/git walking knows about release branches so it will find
> hot-patch versions correctly, as well.
>
In the Bioc case, I mean, not generally.
And with that flurry of emails I'm off to bed.
Best,
~G
>
> ~G
>
> On Thu, Oct 5, 2017 at 10:01 PM, Gabe Becker <beckerg4 at gene.com> wrote:
>
>> Correct. The actual order of checks is:
>>
>> 1. did switchr already retrieve that exact version for something else
>> and keep it around,
>> 2. A GRANRepository object if one is specified (don't worry much
>> about this one)
>> 3. the manifest itself (cran and bioc source types are ignored, but
>> it would walk SCM if it had git/svn type manifest entry for the package)
>> 4. cran repo and cran archives,
>> 5. bioc repositories (all of them in descending order),
>> 6. bioc git (bioc SVN for the version on CRAN, which appeared to
>> still work last time I checked it a week or two ago)
>>
>> See the code in https://github.com/gmbecker/switchr/blob/master/R/
>> retrievePkgVersion.R for details.
>>
>> Best,
>> ~G
>>
>>
>>
>>
>> On Oct 5, 2017 9:06 PM, "Henrik Bengtsson" <henrik.bengtsson at gmail.com>
>> wrote:
>>
>> That's really nice; I didn't know it could do all that. For my
>> clarification, when using PkgManifest(..., type = "bioc") it'll search
>> (i) the CRAN archives, (ii) the Bioconductor repos(es), and then (iii)
>> the Bioconductor Git repos - is that a correct observation? (I
>> installed from https://github.com/gmbecker/switchr)
>>
>> /Henrik
>>
>>
>>
>>
>> On Thu, Oct 5, 2017 at 4:15 PM, Gabe Becker <becker.gabe at gene.com> wrote:
>> > In point of fact, it looks like IRanges 2.6.0 is an instance of that
>> > weakness, so was probably a bad example. 2.6.1 installs correctly, or
>> would
>> > in it's native R/base bioc environment... (it fails for me in the
>> library I
>> > have...)
>> >
>> > Also, the version on CRAN uses the bioc SVN, so may not work for recent
>> > versions.
>> >
>> > On Thu, Oct 5, 2017 at 3:58 PM, Gabe Becker <beckerg4 at gene.com> wrote:
>> >>
>> >> Henrik et al.,
>> >>
>> >> My switchr package (on CRAN, github at:
>> >> http://github.com/gmbecker/switchr, preprint of the paper here:
>> >> https://arxiv.org/abs/1501.02284) can do this.
>> >>
>> >> In fact, installing (cohorts of) old versions of packages is one of
>> it's
>> >> primary purposes. Specifically, it can install old source versions of
>> >> packages from CRAN, Bioconductor, and general Git and SVN repos you
>> tell it
>> >> about.
>> >>
>> >> With the caveat that it's a bad idea in the general case to specify an
>> old
>> >> version of one package without specifying versions of its dependencies
>> >> (switchr allows you to do this via a manifest, which can be
>> constructed from
>> >> sessionInfo output or guessed in the case of a CRAN package), you can
>> just
>> >> do
>> >>
>> >> > man = PkgManifest(name="IRanges", type="bioc")
>> >>
>> >> > install_packages("IRanges", man, versions = c(IRanges = "2.6.0"))
>> >>
>> >>
>> >> And you will successfully completely break your Bioc installation by
>> >> installing IRanges 2.6.0 into it. ;-)
>> >>
>> >> Switchr also gives you tools to more easilly maintain multiple
>> libraries
>> >> which contain, for example, different bioc versions in them.
>> >>
>> >> NB: switchr is subject to the caveat Martin pointed out and will fail
>> to
>> >> retrieve a buildable version of the package if said buildable version
>> is not
>> >> the first commit in SCM bearing that version in its DESCRIPTION file.
>> >>
>> >> Hope that helps.
>> >>
>> >> Best,
>> >> ~G
>> >>
>> >> On Thu, Oct 5, 2017 at 2:21 PM, Martin Morgan
>> >> <martin.morgan at roswellpark.org> wrote:
>> >>>
>> >>> On 10/05/2017 05:14 PM, Henrik Bengtsson wrote:
>> >>>>
>> >>>> On Thu, Oct 5, 2017 at 1:46 PM, Martin Morgan
>> >>>> <martin.morgan at roswellpark.org> wrote:
>> >>>>>
>> >>>>> On 10/05/2017 01:50 PM, Henrik Bengtsson wrote:
>> >>>>>>
>> >>>>>>
>> >>>>>> Is there an easily accessible archive for Bioconductor packages
>> >>>>>> similar to what is provided on CRAN where you can find all released
>> >>>>>> versions of a package, e.g.
>> >>>>>> https://cran.r-project.org/src/contrib/Archive/PSCBS/?
>> >>>>>>
>> >>>>>> Say I want to access the source code for affy 1.18.0. Here are the
>> >>>>>> two approaches I'm aware of and none of them are particularly
>> >>>>>> appealing to me. Does anyone know of a better approach?
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> The only option is to scrape, and that's approximate. One could
>> build
>> >>>>> an
>> >>>>> archive
>> >>>>>
>> >>>>> pkg,version,branch,from_svn_rev,to_svn_rev
>> >>>>>
>> >>>>> and then consult that. Packages are supposed to increment the 'z' of
>> >>>>> x.y.z,
>> >>>>> but I'm sure there are many exceptions. I believe Jim Hester has an
>> svn
>> >>>>> script for this, but I wasn't able to locate it; it would be fast in
>> >>>>> git.
>> >>>>
>> >>>>
>> >>>> Thanks. About 'z' not being increased. Does the Bioc build servers
>> >>>> release (a) continuously or (b) only when it detects a version change
>> >>>> x.y.z -> x.y.z+1? If it does it continuously, then what x.y.z is
>> >>>> installed does matter on when it was downloaded/installed, correct?
>> >>>> On the other hand, if it only builds in when a version bump is
>> >>>> detected, then one can at least narrow it down to a much narrow set
>> of
>> >>>> x.y.z submits (if multiple exists).
>> >>>
>> >>>
>> >>> The builder only pushes for upward increments, so a commit without a
>> >>> 'positive' version bump would be built but not pushed to the public
>> >>> repository. I'm not sure how rigorously this policy was enforced
>> before,
>> >>> e.g., 2005.
>> >>>
>> >>> Of course there are exceptions, e.g., it is occasionally (at most one
>> or
>> >>> two times a release cycle) necessary to flush the public repository
>> >>> entirely, and then whatever is built is pushed. And there is nothing
>> >>> stopping the user from doing a check-out from svn. Perhaps others
>> will chime
>> >>> in with the gory / correct details.
>> >>>
>> >>> Martin
>> >>>
>> >>>
>> >>>>
>> >>>>>
>> >>>>> For your future self, this
>> >>>>>
>> >>>>>
>> >>>>> https://bioconductor.org/packages/3.5/bioc/src/contrib/Archi
>> ve/S4Vectors/
>> >>>>>
>> >>>>> provides a hint of a change coming with the next release --
>> archives of
>> >>>>> all
>> >>>>> RELEASE package versions, starting in Bioc 3.6. (Kudos to Val for
>> >>>>> implementing this)
>> >>>>
>> >>>>
>> >>>> This is great! Thanks Val for this.
>> >>>>
>> >>>> Thanks
>> >>>>
>> >>>> Henrik
>> >>>>
>> >>>>>
>> >>>>> Martin
>> >>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> # APPROACH 1: Download from http://bioconductor.org
>> >>>>>>
>> >>>>>> The best approach I know now is to try to guess the date when this
>> was
>> >>>>>> released in order to identify the Bioconductor release version.
>> >>>>>> Something like this:
>> >>>>>>
>> >>>>>> 1. Guess around 2010.
>> >>>>>>
>> >>>>>> 2. Go to http://bioconductor.org/about/release-announcements/ and
>> see
>> >>>>>> what R versions were in use during 2010. I find R 2.6.x and R
>> 2.7.x.
>> >>>>>> The Bioc version for those R versions (same URL) are Bioc 2.1 and
>> Bioc
>> >>>>>> 2.2. Let's focus on Bioc 2.2 (because I happen to know that is the
>> >>>>>> one)
>> >>>>>>
>> >>>>>> 3. Following the Bioc 2.2 link on above URL to get to
>> >>>>>> http://bioconductor.org/packages/2.2/BiocViews.html.
>> >>>>>>
>> >>>>>> 4. Click through, one eventually gets to
>> >>>>>> http://bioconductor.org/packages/2.2/bioc/html/affy.html
>> >>>>>>
>> >>>>>> 5. The "Source" link points to
>> >>>>>>
>> >>>>>> http://bioconductor.org/packages/2.2/bioc/src/contrib/affy_1
>> .18.2.tar.gz
>> >>>>>>
>> >>>>>> Say I wanted affy 1.16.0 instead and I made the wrong guess in
>> Step 2,
>> >>>>>> I can extrapolate from (Bioc 2.2, affy 1.18.x) finding that I
>> should
>> >>>>>> go to Bioc 2.1 to find affy 1.16.x (because releases have even
>> minor
>> >>>>>> version numbers). It works, but is a bit tedious if you need to
>> do
>> >>>>>> this more than once.
>> >>>>>>
>> >>>>>> Also, I'm pretty sure I read somewhere that Bioc on archive the
>> most
>> >>>>>> recent package version under each release, which means there is no
>> >>>>>> affy_1.18.0.tar.gz available for download. Is that correct?
>> >>>>>>
>> >>>>>>
>> >>>>>> # APPROACH 2: Version control
>> >>>>>>
>> >>>>>> $ git clone https://git.bioconductor.org/packages/affy
>> >>>>>> $ cd affy
>> >>>>>>
>> >>>>>> # Package releases/versions are not tagged
>> >>>>>> $ git tag
>> >>>>>> [empty]
>> >>>>>>
>> >>>>>> # Check Bioc release branches
>> >>>>>> $ git branch -a
>> >>>>>> * master
>> >>>>>> remotes/origin/HEAD -> origin/master
>> >>>>>> remotes/origin/RELEASE_1_0
>> >>>>>> remotes/origin/RELEASE_1_0_branch
>> >>>>>> remotes/origin/RELEASE_1_4
>> >>>>>> remotes/origin/RELEASE_1_4_branch
>> >>>>>> remotes/origin/RELEASE_1_5
>> >>>>>> [...]
>> >>>>>> remotes/origin/RELEASE_3_5
>> >>>>>> remotes/origin/master
>> >>>>>>
>> >>>>>> That's back to above game of trying to narrow down which Bioc
>> release
>> >>>>>> I should look at. A similar approach is to look at the commit log:
>> >>>>>>
>> >>>>>> $ git log DESCRIPTION
>> >>>>>>
>> >>>>>> commit 35573048255b398f99ff1d3560906b2121912248
>> >>>>>> Author: Herve Pages <hpages at fhcrc.org>
>> >>>>>> Date: Mon Apr 24 19:50:57 2017 +0000
>> >>>>>>
>> >>>>>> bump x.y.z versions to odd y after creation of 3_5 branch
>> >>>>>>
>> >>>>>> git-svn-id:
>> >>>>>>
>> >>>>>>
>> >>>>>> file:///home/git/hedgehog.fhcrc.org/bioconductor/trunk/madma
>> n/Rpacks/affy at 129129
>> >>>>>> bc3139a8-67e5-0310-9ffc-ced21a209358
>> >>>>>>
>> >>>>>> commit aa4c2d648658e8c2cca2baf651aea92df55a4392
>> >>>>>> Author: Herve Pages <hpages at fhcrc.org>
>> >>>>>> Date: Mon Apr 24 19:25:24 2017 +0000
>> >>>>>>
>> >>>>>> bump x.y.z versions to even y prior to creation of 3_5 branch
>> >>>>>>
>> >>>>>> git-svn-id:
>> >>>>>>
>> >>>>>>
>> >>>>>> file:///home/git/hedgehog.fhcrc.org/bioconductor/trunk/madma
>> n/Rpacks/affy at 129126
>> >>>>>> bc3139a8-67e5-0310-9ffc-ced21a209358
>> >>>>>>
>> >>>>>> [...]
>> >>>>>>
>> >>>>>> and try to locate affy 1.18.0 by peeking at the DESCRIPTION file
>> >>>>>> history.
>> >>>>>>
>> >>>>>> Does anyone know of a better/more automated approach?
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>>
>> >>>>>> Henrik
>> >>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> Bioc-devel at r-project.org mailing list
>> >>>>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> This email message may contain legally privileged and/or
>> confidential
>> >>>>> information. If you are not the intended recipient(s), or the
>> employee
>> >>>>> or
>> >>>>> agent responsible for the delivery of this message to the intended
>> >>>>> recipient(s), you are hereby notified that any disclosure, copying,
>> >>>>> distribution, or use of this email message is prohibited. If you
>> have
>> >>>>> received this message in error, please notify the sender
>> immediately by
>> >>>>> e-mail and delete this email message from your computer. Thank you.
>> >>>
>> >>>
>> >>>
>> >>> This email message may contain legally privileged
>> and/or...{{dropped:2}}
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> Bioc-devel at r-project.org mailing list
>> >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Gabriel Becker, Ph.D
>> >> Associate Scientist
>> >> Bioinformatics and Computational Biology
>> >> Genentech Research
>> >
>> >
>> >
>> >
>> > --
>> > Gabriel Becker, Ph.D
>> > Associate Scientist
>> > Bioinformatics and Computational Biology
>> > Genentech Research
>>
>>
>>
>
>
> --
> Gabriel Becker, Ph.D
> Associate Scientist
> Bioinformatics and Computational Biology
> Genentech Research
>
--
Gabriel Becker, Ph.D
Associate Scientist
Bioinformatics and Computational Biology
Genentech Research
[[alternative HTML version deleted]]
More information about the Bioc-devel
mailing list