[Bioc-devel] Bioconductor Git/GitHub Mirrors

Leonardo Collado Torres lcollado at jhu.edu
Wed Jun 17 05:21:12 CEST 2015 

On Tue, Jun 16, 2015 at 10:02 PM, Dan Tenenbaum <dtenenba at fredhutch.org> wrote:
>
>
> ----- Original Message -----
>> From: "Leonardo Collado Torres" <lcollado at jhu.edu>
>> To: "Ryan C. Thompson" <rct at thompsonclan.org>
>> Cc: "Dan Tenenbaum" <dtenenba at fredhutch.org>, "bioc-devel" <bioc-devel at r-project.org>
>> Sent: Tuesday, June 16, 2015 5:55:05 PM
>> Subject: Re: [Bioc-devel] Bioconductor Git/GitHub Mirrors
>>
>> Hi,
>>
>>
>> This sounds great! Not having to setup all those bridges everytime a
>> new BioC version is released should be a great feature, plus keeping
>> the commit history tidy. And we can still use Travis or other things
>> like Slack notifications on our own bridges. I guess that it'd be
>> good
>> to remind users to pull new content from the bioc remote before
>> trying
>> to commit to svn.
>>
>> Anyhow, great work!
>>
>>
>>
>>
>> I'm having a few problems with this new setup which hopefully are
>> easy
>> to iron out. Basically, I am stuck at the "git svn rebase && git svn
>> dcommit --add-author-from" step. See 2b) below.
>>
>>
>> 1) My git was pointing to an older version even after updated with
>> brew. This was because I still had MacPorts around.
>> How to diagnose:
>> $ which git
>> Wrong version was pointing to /opt/local/bin/git instead of
>> /usr/local/bin/git Can also see by running
>> $ git --version ## if it doesn't match the version number you
>> installed from brew
>> Solution:
>> * Uninstall all of MacPorts following instructions from
>> https://guide.macports.org/chunked/installing.macports.uninstalling.html
>> * Follow part of
>> http://stackoverflow.com/questions/20648235/error-linking-git-to-homebrew
>> $ sudo brew prune
>> $ brew update
>> $ brew install git
>> $ brew unlink git && brew link git
>
> Good to know. Things seem to work on the Macs I have tested with the vanilla git-svn that ships with the OS.
>

Yeah, I've had mine for a while and forgot I still had MacPorts around.

>
>>
>> You might want to update subversion.
>> http://blog.victorquinn.com/fix-git-svn-in-mountain-lion might be
>> relevant for you.
>>
>
> I haven't needed to update svn.
>
>

Ok, I just did it to be safe.

Link is about those updating to Mountain Lion or Mavericks and having
to re-download Xcode then setting up the command line tools and them
some potential perl issues which I didn't have.

>
>> 2) Test case: derfinderHelper.
>>
>> Smallest and simplest package I have, which won't matter much if
>> things break because it's stable.
>>
>> a) You might want to delete update_remotes.sh after using it.
>> * I deleted the git-svn bridges to both
>> https://github.com/leekgroup/derfinderHelper and
>> https://github.com/leekgroup/derfinderHelper-release
>> * On https://github.com/leekgroup/derfinderHelper I removed bioc as a
>> collaborator and deleted the push-hook since it seems like we don't
>> need this anymore
>
> Yes, you can delete the push hook. Leaving it does no harm as long as the bridge has been deleted (nobody will listen to the hook on the other end once the bridge is deleted).

Ok!

>
>
>> * Download and ran update_remotes.sh as instructed. It doesn't seem
>> like we need it anymore after that point. If you leave it there, it
>> seems to give problems.
>
> Can you explain what problems?

Git complains that you have an untracked file in the repo when running
git svn rebase. There's no need to actually track it, or even exclude
it with a .gitignore file, so I simply deleted it. I did check that
the remotes had been added using "git remote -v".

>
>> Easy to solve, but might want to clarify in the instructions.
>
> I do this in a separate directory:
>
> git clone https://github.com/Bioconductor/mirror.git
>
> and then I have a repos containing the script where I can run 'git pull' if it ever changes. You could also symlink to it in /usr/local/bin. We'll see about changing the docs for this.
>
>>
>> b) Following the section "Use Git Locally And Publicly" I get stuck
>> on
>> the "git svn rebase && git svn dcommit --add-author-from" step as
>> shown at https://gist.github.com/lcolladotor/ac8838b25955caeebcf1.
>> The
>> "CONFLICT (add/add): Merge conflict" message lead me to
>> http://stackoverflow.com/questions/19475387/how-to-handle-fix-git-add-add-conflicts
>> from where I could resolve the initial problem that I saw using "git
>> diff". But was basically the changes between the very first version
>> of
>> README.md and the very latest version of it.
>> There must be a way to not have to go through every single commit
>> and/or file one by one. Using "git rebase --continue" doesn't help.
>>
>
>
> I didn't have this problem. Here is what I did, which should be the same as what you did:
>
> forked derfinderHelper to my personal (dtenenba) account

 Ok, maybe this is why you don't have the problem. I didn't fork
https://github.com/Bioconductor-mirror/derfinderHelper into my
personal (or organization, doesn't matter here) account. I did all of
this with my local clone of
https://github.com/leekgroup/derfinderHelper which already existed.
https://github.com/leekgroup/derfinderHelper has the full commit
history since I've been using git from the beginning, instead of the
aggregated svn commit history.

If I fork https://github.com/Bioconductor-mirror/derfinderHelper then
I'll lose the detailed commit history, right?

In terms of the deprecated git-svn bridge, I want git to win
unconditionally on the merge. I'm not sure how I would do that.


Another reason why I would like to avoid doing a fork is that several
users already "watch", starred the original github repo, or created
issues (some might be open still, others closed with info others might
want to read). Well, not really with the tiny derfinderHelper. But I
guess that this is just really a weak argument since I could just let
svn win while preserving the original github repo.


> cloned the fork
> ran the update_remotes.sh script in the fork
>
> git checkout devel
> git svn rebase
> git merge master --no-edit
> git svn rebase
> git svn dcommit --add-author-from
>
> None of these lines gave any errors and the last line said:
>
> Committing to https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/derfinderHelper ...
>
> Doesn't seem like it actually needed to commit anything.
>
>> c) Given the issue with b, the devel branch already has a merge
>> commit. Using
>> http://stackoverflow.com/questions/927358/how-to-undo-the-last-commit
>> I cleaned things up. I don't think it's needed, but I don't want to
>> run into more issues.
>> $ git rebase --abort
>> $ git reset HEAD~1
>>
>>
>> As you can tell, I'm no "git rebase" expert. From
>> http://git-scm.com/docs/git-rebase it seems like the "ours" option
>> might be useful. But right now I prefer to get some pointers on how
>> to
>> proceed before playing around further :P
>
>
> Well, since I can't reproduce this situation I'm not exactly sure what to tell you. If you know you want all the changes from git and none from svn (or vice versa) then the --ours and --theirs options are appropriate. But you might just want to start over and see if the problem goes away.

Ahh, ok. Sounds like what I want!

http://stackoverflow.com/questions/2959443/why-is-the-meaning-of-ours-and-theirs-reversed-with-git-svn
seems like a good starting point

>
> Here are my relevant versions:
>
> $ git --version
> git version 1.9.5 (Apple Git-50.3)
> $ git svn --version
> git-svn version 1.9.5 (Apple Git-50.3) (svn 1.7.19)
> $ svn --version
> svn, version 1.8.11 (r1643975)
> $ uname -a
> Darwin viper 13.4.0 Darwin Kernel Version 13.4.0: Wed Mar 18 16:20:14 PDT 2015; root:xnu-2422.115.14~1/RELEASE_X86_64 x86_64
>
> It does look like on this particular machine I have a non-standard svn:
>
> $ which svn
> /opt/subversion/bin/svn
>
>
> Looks like it came from: https://www.wandisco.com/subversion/os/downloads
>
>
>
>>
>> Thanks,
>> Leo
>>
>>
>> PS At the "Unable to determine upstream SVN information" section, I
>> believe that there is a typo. "git svn dcomit" should be "git svn
>> dcommit"
>
> Will fix.
>
> Thanks for the comments and for trying it out.
>
> Dan
>
>>
>> On Tue, Jun 16, 2015 at 3:20 PM, Ryan C. Thompson
>> <rct at thompsonclan.org> wrote:
>> > This is great to hear. I sometimes want to delve into the source
>> > code of a
>> > package's internals, but doing so through the SVN web interface is
>> > clunky.
>> > Being able to use Github's repo browsing functionality for Bioc
>> > packages is
>> > great.
>> >
>> >
>> > On 06/16/2015 12:00 PM, Dan Tenenbaum wrote:
>> >>
>> >> Dear Bioconductors,
>> >>
>> >> We're pleased to announce the availability of Bioconductor Git
>> >> Mirrors.
>> >> These are read-only GitHub repositories (available under
>> >> https://github.com/Bioconductor-mirror)
>> >> for every Bioconductor software package. These repositories are
>> >> synchronized with our
>> >> Subversion repository. Package maintainers (or anyone else) can
>> >> fork these
>> >> repositories
>> >> and do their development on the fork. Complete documentation of
>> >> the
>> >> mirrors is at
>> >>
>> >> http://bioconductor.org/developers/how-to/git-mirror/
>> >>
>> >> These mirrors supersede the Git-SVN bridge, which is now
>> >> deprecated.
>> >> Creation of new bridges is disabled and maintainers who are using
>> >> the
>> >> bridge
>> >> should migrate to the Git mirrors as soon as it's convenient, as
>> >> the
>> >> bridge
>> >> will eventually go away. Instructions for migrating can be found
>> >> at the
>> >> above link.
>> >>
>> >> Some features of the new Git Mirrors, and why we feel they are a
>> >> better
>> >> solution than the Git-SVN bridge:
>> >>
>> >>   - The mirrors contain complete commit history.
>> >>   - The mirrors contain release branches for Bioconductor 3.0 and
>> >>   3.1, and
>> >>     new releases will be added as they happen. You will no longer
>> >>     need
>> >>     separate repositories for release and devel.
>> >>   - Setup is easy and you no longer have to grant any permissions
>> >>   on your
>> >>     repository to other users. You can commit directly to
>> >>     Subversion
>> >>     using git-svn (https://git-scm.com/docs/git-svn).
>> >>   - Each git commit appears in the SVN log as a distinct SVN
>> >>   commit;
>> >> commits
>> >>     are no longer grouped together as they were with the bridge.
>> >>     Each git
>> >>     commit can be mapped to a specific svn commit, and vice versa.
>> >>   - The Git mirrors are much more reliable.
>> >>   - Use is flexible. You can use git locally (without GitHub) or
>> >>   you
>> >>     can use GitHub as well, to take advantage of all its social
>> >>     coding
>> >> features.
>> >>   - Using GitHub's code search, you can search the entire
>> >>   Bioconductor
>> >> codebase.
>> >>     Here's a sample search: https://goo.gl/jI92Ys
>> >>   - Subversion is fully supported and remains the cannonical
>> >>   repository;
>> >> use
>> >>     of Git and GitHub is optional.
>> >>
>> >> We are excited about these new mirrors (brought to you by the hard
>> >> work of Jim Hester) and we hope you are too. Questions and
>> >> comments are
>> >> welcome on the bioc-devel mailing list.
>> >>
>> >> Dan
>> >>
>> >> _______________________________________________
>> >> Bioc-devel at r-project.org mailing list
>> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >
>> >
>> > _______________________________________________
>> > Bioc-devel at r-project.org mailing list
>> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>

More information about the Bioc-devel mailing list