[Bioc-devel] GitHub and svn

Sun Oct 16 07:25:23 CEST 2016

Hi folks, I'm a person who prefers to use Git as my main source for my
Bioc packages.  These Git repositories are then synced online to
GitHub for each collaboration yada yada.  This also means that these
days I also tend to think of the Bioconductor SVN repository mostly as
a communication channel for submitting new versions (very much as you
FTP package updates to CRAN).

That's my setup.  Now, I constantly failed to remember the git svn
commands to synchronizing between local Git repository and the Bioc
SVN repository - to be honest - it was lots of trial and error
whenever I had to do it.  This caused me to hold back on committing
updates or sometimes I committed directly to the Bioc SVN and manually
copied it over to the Git repository.  Things didn't work smoothly
because I constantly ran into those dreaded `git svn` errors that I
think is due to a non-linear Git history or whatever.

So, I wrote the git-bioc tools: https://github.com/HenrikBengtsson/git-bioc

Now, when I've done updates to my Git repository and feel they're
ready to go live on Bioc devel (after first having pushed to GitHub
and verified that Travis CI and Appveyor CI reports all OK), I
basically do:

1. git bioc pull
2. git bioc import (and solve any conflicts, iff needed)
3. git bioc export (and solve any conflicts, iff needed)
4. git bioc push

The first two won't do anything unless there have been updates
directly to the SVN repos.

Feel free to try it out, improve on it and so on.  I don't have any
plans to maintain this for everyone, but maybe it's a good enough stub
for someone else to build something more general.  Things that can be
improved is how commit messages are compiled / generated and how
conflicts are solved.

Before using it, please read the big WARNING in the README file.

/Henrik

On Sat, Oct 15, 2016 at 6:05 PM, Nathan Sheffield
<nathan at code.databio.org> wrote:
> Hi Kevin, Mani --
>
> Isn't the git-svn bridge deprecated?
>
> http://bioconductor.org/developers/how-to/git-svn/
> And this message:
>
> ----- Forwarded Message -----
> From: "Dan Tenenbaum"<dtenenba at fredhutch.org>
> To: "bioc-devel"<bioc-devel at r-project.org>
> Sent: Monday, December 28, 2015 11:08:29 AM
> Subject: [Bioc-devel] Git-Svn bridge will be removed permanently on January
> 29, 2016
>
> Attention Bioconductor developers,
>
> The deprecated Git-SVN bridge will be removed permanently on Friday, January
> 29th, 2016.
>
> But anyway, Mani, I at least agree with you that the current system is
> "unnecessarily complicated" (git-svn, cherry-picking, credentials, mirrors
> and so forth) and I hope it will become much better after, as Kasper put it,
> "the (long) process of changing this" gets finished, which I hope means a
> system that is git-centered, since that seems to be the way the world has
> moved.
>
> I, for one at least, have several ideas for new packages and maintenance of
> old ones that I continue to put off on the hopes that one day soon the
> update system will be as easy for an official Bioconductor repo as it is to
> simply push a git commit to my own repository on GitHub. But alas, it is not
> yet the case -- but I think when this gets "fixed" it will be a glorious
> future indeed.
>
> -Nathan
>
>
> On 10/15/2016 06:41 PM, Kevin RUE wrote:
>>
>> Dear Mani,
>>
>> That's actually where I think the git-svn bridge becomes useful.
>> If you take the time once to synchronise the devel branch of your package
>> from the Bioconductor-mirror to one of the branch on your GitHub
>> repository
>> (for me the master branch), you could then:
>> 1) commit changes to your own repository first (does not affect the BioC
>> code)
>> 2) follow the BioC git-svn instructions that you mentioned to selectively
>> push the changes to either or both the devel and release branche(s) of
>> Bioconductor.
>>
>> I personally don't think that the current system is "unnecessarily
>> complicated". That handful of commands is probably as simple as a system
>> can be, to handle version control selectively applied to multiple branches
>> (devel, 3.3, 3.2, master, ...) of multiple repositories (BioC, GitHub,
>> ...)
>> using multiple version control software (git, svn).
>>
>> The learning curve for version control can be quite steep beyond the basic
>> commands, but one has to bear in mind that RStudio is not a version
>> control
>> software in itself. The GUI only provides buttons for the most common
>> version controls commands (pull/push/update). For more advanced commands,
>> you will have to open a shell anyway (the little wheel icon).
>> I must admit that I was a bit scared/frustrated at first with the system,
>> but after a few attempts to get it right, this little process to control
>> the branches to synchronise can almost become enjoyable when proudly
>> releasing new code :)
>>
>> All the best,
>> Kevin
>>
>>
>> On Sat, Oct 15, 2016 at 10:21 PM, S Manimaran <manimaran_1975 at hotmail.com>
>> wrote:
>>
>>> Thanks, Gabe and Kasper, for the info.
>>>
>>>
>>>
>>> Following up on Gabe's reply: That means, I need to setup two projects in
>>> R-Studio with one for the release pointing to the release repository in
>>> SVN
>>> and another for the development version pointing to the development
>>> repository in SVN, right? Now, suppose I make a bug fix and commit to the
>>> release repository and I want the same fix in the development repository
>>> as
>>> well, how exactly do I go about this: Do I just manually copy those files
>>> with the changes to the other development version project and commit it
>>> there as well? (Personally, I also like to keep the original GitHub
>>> repository in sync with the latest in BioConductor development, which
>>> would
>>> mean I need to maintain three projects in R-Studio, right?) Or is there
>>> any
>>> other way about this?
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Mani
>>>
>>>
>>>
>>>
>>> From: Gabe Becker [mailto:becker.gabe at gene.com]
>>> Sent: Saturday, October 15, 2016 4:35 PM
>>> To: Kasper Daniel Hansen
>>> Cc: S Manimaran; bioc-devel at r-project.org
>>> Subject: Re: [Bioc-devel] GitHub and svn
>>>
>>>
>>> Mani,
>>>
>>> Related to what Kasper said, one thing you can do is commit directly to
>>> the canonical repo for your package (which again is not on github once
>>> the
>>> package is accepted) from rstudio. It supports svn.
>>>
>>> ~G
>>>
>>> On Oct 15, 2016 11:38 AM, "Kasper Daniel Hansen" <
>>> kasperdanielhansen at gmail.com<mailto:kasperdanielhansen at gmail.com>> wrote:
>>> Not at the moment.  We are in the (long) process of changing this, but
>>> there is no ETA for it.
>>>
>>> The complications we currently have, as soon as a package is accepted in
>>> Bioconductor, is that the "true" repository then becomes Bioconductor SVN
>>> and your Github repository is just a way for you to develop.  This is not
>>> the case during package submission.
>>>
>>> Best,
>>> Kasper
>>>
>>>
>>> On Sat, Oct 15, 2016 at 12:19 PM, S Manimaran
>>> <manimaran_1975 at hotmail.com<
>>> mailto:manimaran_1975 at hotmail.com>>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I never understood the github mirror setup and the instructions below
>>>
>>> look
>>>>
>>>> unnecessarily complicated to me. I see that the current package
>>>
>>> submission
>>>>
>>>> process with the automatic hook added to github is the most easiest of
>>>
>>> all
>>>>
>>>> with every commit to github automatically triggering a build at
>>>> Bioconductor. Now, my question is: Can't this same procedure be carried
>>>> over once the Bioconductor 3.4 is released as well i.e commits to github
>>>> automatically resulting in triggering a build at Bioconductor? The main
>>>
>>> use
>>>>
>>>> case that I am looking for is an easy way to commit directly from inside
>>>> R-Studio. With R-Studio setup for GitHub project, it directly commits to
>>>> GitHub, but now for having to commit to BioConductor, if the automatic
>>>> trigger works well as is the case with the new package submission
>>>
>>> process,
>>>>
>>>> all is well and good. But if I have to do as what the page in git-mirror
>>>> says, then it looks like that I have to get out of R-Studio to do some
>>>> overly complicated process to achieve the s!
>>>>   ame. It will be really helpful if I can continue to use the automatic
>>>> trigger to automatically build after Bioconductor 3.4 release as well.
>>>>
>>>>
>>>> http://bioconductor.org/developers/how-to/git-mirror/
>>>>
>>>> Scenario 2: Set Up Your Own GitHub Repository
>>>> If you do not already have a public git repository for package REPO the
>>>> simplest thing to do is navigate to https://github.com/
>>>> Bioconductor-mirror/REPO and click the Fork button in the upper right.
>>>> This will create a copy of the repository on your personal account. You
>>>
>>> may
>>>>
>>>> want to re-enable issue tracking in your repository (it's disabled in
>>>> the
>>>> read-only mirrors and forks inherit this setting). To do this, go to
>>>> Settings and then click the Issues checkbox. Then perform the following
>>>> steps in your terminal.
>>>>
>>>>    1.  git clone https://github.com/USER/REPO to clone the repository to
>>>> your machine.
>>>>    2.  cd REPO to switch to the REPO directory.
>>>>    3.  bash /path/to/update_remotes.sh to setup the git remotes.
>>>>    4.  Commit to git and push to GitHub as you normally would.
>>>>    5.  Each time you want to push git commits to svn:
>>>>       *   git checkout devel to switch to the devel branch. (use
>>>> release-X.X for release branches)
>>>>       *   git svn rebase to get the latest SVN changes.
>>>>       *   git merge master --log to merge your changes from the master
>>>> branch or skip this step and work directly on the current branch.
>>>>       *   git svn dcommit --add-author-from to sync and commit your
>>>
>>> changes
>>>>
>>>> to svn. You may be prompted here for your SVN username and password.
>>>> When you're done, be sure and merge any changes from svn back into the
>>>
>>> git
>>>>
>>>> master branch:
>>>> git checkout master
>>>> git merge devel
>>>>
>>>> Thanks,
>>>> Mani
>>>>
>>>
>>>
>>>          [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> Bioc-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioc-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel