[Bioc-devel] Bioconductor Git/GitHub Mirrors
davide risso
risso.davide at gmail.com
Sat Jun 20 05:07:29 CEST 2015
Hi all,
I don't know why but this is not working for me.
I deleted the bridge for my RUVSeq package.
I forked Bioconductor-mirror/RUVSeq into drisso/RUVSeq-mirror
I then run:
$ git clone https://github.com/drisso/RUVSeq-mirror
$ bash ../update_remotes.sh
$ git checkout devel
error: pathspec 'devel' did not match any file(s) known to git.
$ git branch
* master
I tried the same with my other package EDASeq and again it does not work.
Am I doing something wrong? It seems that the issue is with the
update_remotes script, but I cannot figure out what's wrong.
Best,
Davide
On Fri, Jun 19, 2015 at 1:52 PM Leonardo Collado Torres <lcollado at jhu.edu>
wrote:
> Hi,
>
> Dan previously said:
>
> Try starting over again. Remove your local repository and do a fresh clone:
>
> git clone https://github.com/leekgroup/derfinderHelper.git
> cd derfinderHelper
> bash /path/to/update_remotes.sh
> git checkout devel
> git svn rebase
> git merge master --no-edit
> git svn dcommit --add-author-from
>
>
>
> So I went and did that. Note that if you have local branches, like I
> did, make sure that you push them to GitHub first.
>
>
> 1) I first encountered an authentification error. I solved it below by
> deleting any previous SVN auth info I had (I only use SVN for Bioc).
>
> $ git svn dcommit --add-author-from --username=l.collado-torres
> Committing to
> https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/derfinderHelper
> ...
>
> ERROR from SVN:
> URL access forbidden for unknown reason: Access to
> '/bioconductor/!svn/me' forbidden
> No changes between ef1c7dc3236ba31adace8c2205d30cc76e913032 and
> refs/remotes/git-svn-devel
> Resetting to the latest refs/remotes/git-svn-devel
> ERROR: Not all changes have been committed into SVN, however the committed
> ones (if any) seem to be successfully integrated into the working tree.
> Please see the above messages for details.
>
> Solution:
>
> ## Check what auth files I had
> $ ls ~/.subversion/auth/svn.simple/*
>
> ## Was my username in any of them? Nope
> $ grep -l l.collado-torres ~/.subversion/auth/svn.simple/*
>
> ## Clear them. If you use SVN for multiple repos, you don't want to do
> this.
> $ rm ~/.subversion/auth/svn.simple/*
>
> ## Yup, it's empty.
> $ ls ~/.subversion/auth/svn.simple/*
>
> ## Start over and use this line on the first git svn rebase
> $ git svn rebase --username=l.collado-torres
>
> ## Now my SVN auth info is there
> $ ls ~/.subversion/auth/svn.simple/*
> $ grep -l l.collado-torres ~/.subversion/auth/svn.simple/*
>
>
>
>
> 2) The commands Dan posted before worked just like he described. That
> is, it didn't commit anything back to SVN (since there's nothing new).
> I'm re-posting them for clarity.
>
> ## Remove local repository and do a fresh clone
> git clone https://github.com/leekgroup/derfinderHelper.git
> cd derfinderHelper
> bash /path/to/update_remotes.sh
> git checkout devel
> git svn rebase
> git merge master --no-edit
> git svn dcommit --add-author-from
>
> ## Output from last command
> Committing to
> https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/derfinderHelper
> ...
> No changes
> 4a21725268e160061103dadb063d0f382235d7e6~1 ==
> 4a21725268e160061103dadb063d0f382235d7e6
> No changes between 4a21725268e160061103dadb063d0f382235d7e6 and
> refs/remotes/git-svn-devel
> Resetting to the latest refs/remotes/git-svn-devel
>
>
> As Dan pointed out earlier, I do not want to make my "master" commit
> history the one from SVN. Want I want to achieve is to have my past
> commit history be the one from GitHub, and then after that, I don't
> mind the double commit messages like in
>
> https://github.com/rmflight/categoryCompare/commit/762bf7046931096b730e21d8a9c4d0d02c734602
> and
> https://github.com/rmflight/categoryCompare/commit/e132385d0ada77bb9a43e1f7e0027f6b6d59af19
>
>
> 3) With my fresh clone, if I follow the original instructions, I still
> end with the problem I described in my first email.
>
>
> $ git clone https://github.com/leekgroup/derfinderHelper.git
> Cloning into 'derfinderHelper'...
> remote: Counting objects: 387, done.
> remote: Total 387 (delta 0), reused 0 (delta 0), pack-reused 387
> Receiving objects: 100% (387/387), 81.50 KiB | 0 bytes/s, done.
> Resolving deltas: 100% (214/214), done.
> Checking connectivity... done.
> cd derfinderHelper
>
> $ bash ../update_remotes.sh
> Commit to git as normal, when you want to push your commits to svn
> 1. `git checkout devel` to switch to the devel branch. (use release-X.X
> for
> release branches)
> 2. `git svn rebase` to get the latest SVN changes.
> 3. `git merge master --no-edit` to merge your changes from the master
> branch
> or skip this step and work directly on the current branch.
> 4. `git svn rebase && git svn dcommit --add-author-from` to sync and
> commit
> your changes to svn.
>
> $ git status
> On branch master
> Your branch is up-to-date with 'origin/master'.
> nothing to commit, working directory clean
>
> $ git checkout devel
> Switched to branch 'devel'
> Your branch is up-to-date with 'bioc/master'.
>
> $ git svn rebase
> Migrating from a git-svn v1 layout...
> Data from a previous version of git-svn exists, but
> .git/svn
> (required for this version (2.4.3) of git-svn) does not exist.
> Done migrating from a git-svn v1 layout
> Rebuilding
> .git/svn/refs/remotes/git-svn-devel/.rev_map.bc3139a8-67e5-0310-9ffc-ced21a209358
> ...
> r94975 = 89b8467a9e02d9f1d8ad2396c8d64cd44babd726
> r95029 = 2372c1563d70030f278f2bf265d83e3013e67611
> r95439 = d781eefa74f756c74561adf95bb638aa90935cc6
> r95443 = 80527e2b9c1442e4d60127fd6d3f8a84cdb4a5db
> r95451 = cf659b64cca6bac275f7c15f93c9ab1407800aa7
> r95474 = 4f5d77e2ea1427d0dbb4a4ffad163f7ad58facc4
> r95475 = 31f6cf4046778942c9357a6378928652559217ff
> r96122 = e02c586ea34f902b4c068bcbe1d2d10d0830a00b
> r96159 = ed98baccbec23e0f70d8aba6a1eb11941c844081
> r96165 = f56cb64e2fbd0cb7e4580047c1f27eb8400db67e
> r96168 = d8b8901ccfa13d30b8e9ecc5b195e7f6842a2194
> r96450 = c5f4ad07ff97d733f36a1f2a6556fdbab7c46459
> r96458 = 2f027fc7ee011b579bc51cc6bb062987fafb16cd
> r96463 = 97f90df0ec9cf1a6131bfd4479c6cca0981fa72c
> r96466 = a193e8c3ca4650a87b0f61416f0cd32169c193df
> r96488 = e3bb06e026e8890e9161d6504e208273998d74c5
> r97003 = f2be8f26f80b369f14df9cd4db37b573db71a8e3
> r97073 = 2698148110d3b45ef335e3ce1ce555af7459bbaa
> r100624 = 4cc67a757cd51ce529575197ab7dddcf7dfde286
> r102591 = 9b856813fa041972a398ddc22362bf3c52789d5b
> r102594 = 441927a8a2d41ea0c7f0db7f05e1c2600a45fe0d
> r104736 = 601760f4c3fffbd897fa4f8fa3a631dbf8d0dda8
> r104883 = 1a2fcc1b1a63b3e7398e5d9001f6c358369d6161
> r104889 = cb528993af8e7d9d4d41a17027dc64aec84e0255
> r104895 = ace63767281447e9497876c700f3538f28d238d9
> r104898 = 08ec97478d7836265db955f3032c28490b212488
> r104900 = b452efe2f8923c5f2229feecc1b399d56364d52d
> r104916 = 3e54179b27b9641cff04f699fb5b915e45347b94
> r104998 = 3e8de0408d30bea6b06163a63d179dec5456eaa8
> Done rebuilding
>
> .git/svn/refs/remotes/git-svn-devel/.rev_map.bc3139a8-67e5-0310-9ffc-ced21a209358
> Current branch devel is up to date.
>
> $ git merge master --no-edit
> Merge made by the 'recursive' strategy.
>
> $ git status
> On branch devel
> Your branch is ahead of 'bioc/master' by 73 commits.
> (use "git push" to publish your local commits)
> nothing to commit, working directory clean
>
> $ git svn rebase && git svn dcommit --add-author-from
> First, rewinding head to replay your work on top of it...
> Applying: Initial commit
> Using index info to reconstruct a base tree...
> Falling back to patching base and 3-way merge...
> Auto-merging README.md
> CONFLICT (add/add): Merge conflict in README.md
> Failed to merge in the changes.
> Patch failed at 0001 Initial commit
> The copy of the patch that failed is found in:
>
> /Users/lcollado/Dropbox/JHSPH/Code/derfinderHelper/.git/rebase-apply/patch
>
> When you have resolved this problem, run "git rebase --continue".
> If you prefer to skip this patch, run "git rebase --skip" instead.
> To check out the original branch and stop rebasing, run "git rebase
> --abort".
>
> rebase refs/remotes/git-svn-devel: command returned error: 1
>
> $ git status
> rebase in progress; onto 3e8de04
> You are currently rebasing branch 'devel' on '3e8de04'.
> (fix conflicts and then run "git rebase --continue")
> (use "git rebase --skip" to skip this patch)
> (use "git rebase --abort" to check out the original branch)
>
> Changes to be committed:
> (use "git reset HEAD <file>..." to unstage)
>
> new file: LICENSE
>
> Unmerged paths:
> (use "git reset HEAD <file>..." to unstage)
> (use "git add <file>..." to mark resolution)
>
> both added: README.md
>
> $ git rebase --abort
>
> $ git status
> On branch devel
> Your branch is ahead of 'bioc/master' by 73 commits.
> (use "git push" to publish your local commits)
> nothing to commit, working directory clean
>
> $ git svn dcommit --add-author-from
> Committing to
> https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/derfinderHelper
> ...
> No changes
> 750d5e459ae6f882922144a6ba4641cdc8ba84b3~1 ==
> 750d5e459ae6f882922144a6ba4641cdc8ba84b3
> No changes between 750d5e459ae6f882922144a6ba4641cdc8ba84b3 and
> refs/remotes/git-svn-devel
> Resetting to the latest refs/remotes/git-svn-devel
> $ git status
> On branch devel
> Your branch is up-to-date with 'bioc/master'.
> nothing to commit, working directory clean
>
>
> 4) In an attempt to try to achieve my goal, I go back to the state
> after the merge (that is, after one git svn rebase). That is:
>
> ## Remove local repository and do a fresh clone
> git clone https://github.com/leekgroup/derfinderHelper.git
> cd derfinderHelper
> bash /path/to/update_remotes.sh
> git checkout devel
> git svn rebase
> git merge master --no-edit
>
> I know the second git svn rebase creates the conflicts, but I thought
> from
> http://stackoverflow.com/questions/2959443/why-is-the-meaning-of-ours-and-theirs-reversed-with-git-svn
> that I could get git svn to recursively choose my git history over my
> svn history. However, it doesn't work as shown below. From
> http://git-scm.com/docs/git-svn, under the "rebase" command, it says
> "This accepts all options that git svn fetch and git rebase accept."
> Below I try a couple of different syntaxes which I thought would work
> given http://git-scm.com/docs/git-rebase
>
> $ git svn rebase -s strategy recursive -X theirs
> Unknown option: X
> $ git svn rebase -s strategy "recursive" -X "theirs"
> Unknown option: X
> $ git svn rebase -s strategy recursive -Xtheirs
> Unknown option: X
> Unknown option: t
> Unknown option: e
> $ git svn rebase -s strategy recursive --strategy-option theirs
> Unknown option: strategy-option
>
> $ git svn rebase -s strategy "recursive -X theirs"
> First, rewinding head to replay your work on top of it...
> /usr/local/Cellar/git/2.4.3/libexec/git-core/git-rebase--merge: line
> 70: git-merge-strategy: command not found
> Unknown exit code (127) from command: git-merge-strategy
> ed5ceda167117937ea498d8713361c34e2fb84e3^ -- HEAD
> ed5ceda167117937ea498d8713361c34e2fb84e3
> rebase --strategy=strategy refs/remotes/git-svn-devel: command returned
> error: 1
> $ git rebase --abort
>
>
> $ git svn rebase -s theirs
> First, rewinding head to replay your work on top of it...
> /usr/local/Cellar/git/2.4.3/libexec/git-core/git-rebase--merge: line
> 70: git-merge-theirs: command not found
> Unknown exit code (127) from command: git-merge-theirs
> ed5ceda167117937ea498d8713361c34e2fb84e3^ -- HEAD
> ed5ceda167117937ea498d8713361c34e2fb84e3
> rebase --strategy=theirs refs/remotes/git-svn-devel: command returned
> error: 1
> $ git rebase --abort
>
> I know
> https://github.com/leekgroup/derfinderHelper/commit/ed5ceda167117937ea498d8713361c34e2fb84e3
> is the first commit in my git history and
>
> https://github.com/Bioconductor-mirror/derfinderHelper/commit/3e8de0408d30bea6b06163a63d179dec5456eaa8is
> the last commit in the svn history.
>
>
> What frustrates me the most at this point is that the opposite
> strategy ("ours") works:
>
>
> ## Remove local repository and do a fresh clone
> git clone https://github.com/leekgroup/derfinderHelper.git
> cd derfinderHelper
> bash /path/to/update_remotes.sh
> git checkout devel
> git svn rebase
> git merge master --no-edit
>
> $ git svn rebase -s ours
> First, rewinding head to replay your work on top of it...
> Already applied: 0001 Initial commit
> Already applied: 0002 Delete LICENSE
> Already applied: 0003 Imported derfinder::fstats.apply(). Having the
> function in a separate package should speed up the permutation cycles
> when using BiocParallel::SnowParam()
> Already applied: 0004 minor formatting changes
> Already applied: 0005 Fix links
>
> ## more output
>
> Already applied: 0066 Change to leekgroup in vignettes
> Already applied: 0067 Make startup messages very rare
> Already applied: 0068 Fix to make messages very rare
> All done.
>
>
> Inspecting with "git log", I only see commits from the svn history.
> The opposite of what I wanted, and hence why I want to use "theirs".
> Given the syntax that works with "ours", I would expect
> $ git svn rebase -s theirs
> to work. Hm...
>
>
> That lead me to
>
> http://stackoverflow.com/questions/173919/is-there-a-theirs-version-of-git-merge-s-ours
> but it didn't apply to this problem because that's for merging, not
> rebasing. During all of this, I noticed that I could manually resolve
> the conflicts using "theirs" via the SourceTree app. Sadly, I don't
> know how to get the command SourceTree runs behind the scene but it
> lead me to believe that I could manually choose for all the
> conflicting files to use "theirs", then add them, then continue
> rebasing, etc. That's how I found
>
> http://stackoverflow.com/questions/24743769/git-resolve-conflict-using-ours-theirs-for-all-files
>
> and...
>
>
> ## Remove local repository and do a fresh clone
> git clone https://github.com/leekgroup/derfinderHelper.git
> cd derfinderHelper
> bash /path/to/update_remotes.sh
> git checkout devel
> git svn rebase
> git merge master --no-edit
> git svn dcommit --add-author-from
> git svn rebase
>
> ## For each conflict loop, do
> grep -lr '<<<<<<<' . | xargs git checkout --theirs
> git add * ## If at some point you added files starting with .
> ## you will have to add them manually
> git rebase --continue
>
> ## Once that's done, do
> git svn dcommit --add-author-from
>
> ## Then pull from the Bioc mirror
> git pull
>
>
>
> In my case, that did:
>
> $ git svn dcommit --add-author-from
> Committing to
> https://hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/derfinderHelper
> ...
> A LICENSE
> M README.md
> Committed r105182
> A LICENSE
> M README.md
> r105182 = 07e2529c91e3bd6f12fc798ac39e4120a4f8154b
> (refs/remotes/git-svn-devel)
> D LICENSE
>
> ## more output
>
> Committed r105249
> M R/zzz.R
> r105249 = d35a0af365c1b7417564b598611371acd3dd7469
> (refs/remotes/git-svn-devel)
> No changes between 9fa56b177a184b18f75c8636a758ce18770831df and
> refs/remotes/git-svn-devel
> Resetting to the latest refs/remotes/git-svn-devel
>
> $ git status
> On branch devel
> Your branch is ahead of 'bioc/master' by 68 commits.
> (use "git push" to publish your local commits)
> nothing to commit, working directory clean
> $ git pull
> >From https://github.com/Bioconductor-mirror/derfinderHelper
> 3e8de04..d35a0af master -> bioc/master
> Already up-to-date.
>
>
>
>
> And now https://github.com/Bioconductor-mirror/derfinderHelper looks
> like I wanted it to look. It has a single SVN commit per git commit I
> originally did. Plus the original commits made directly to SVN by the
> git-svn bridge and others at Bioconductor.
>
> Yay!!!
>
>
>
>
>
> 5) Everything should be in sync.... well, kind of.
>
>
> So I went back to my master branch, did a new commit to test things.
> Then...
>
> git checkout devel
> git svn rebase
> git merge master ## I had some conflicts to resolve here
> git svn rebase
>
> Let me back to that issue with the first commit. Oh no!!
>
>
> However, Dan's earlier solution did work. That is, after making a new
> commit on the master branch
>
>
> git checkout devel
> git svn rebase
> git merge master
> git svn dcommit --add-author-from
>
>
>
> Or it did work partially since
>
> https://github.com/Bioconductor-mirror/derfinderHelper/commit/402100dc8d78ccbb99a4d65dc6f748fb7474a69d
> does not include the message from
>
> https://github.com/leekgroup/derfinderHelper/commit/4b874a895701fe81c108cb4c98d07d0177ebcacd
>
>
>
>
> At this point, I think the best for me would be to make my "master"
> branch have the same history as "devel" (aka master from
> https://github.com/Bioconductor-mirror/derfinderHelper) (my git
> history is there, just under the svn style of commits and with a
> different username) and continue from there like
> https://github.com/rmflight/categoryCompare is doing. Before doing so,
> I'll save my current "master" into another branch and push it to
> GitHub, just as a backup.
>
> Otherwise, for every new commit, I'll have to re-write the full SVN
> commit history and that will take too long.
>
>
>
>
> Ugh... that was huge.
>
>
>
> In summary, if this is ok, I plan for each of my other pkgs to:
>
>
> ## Make a copy of the "master" branch
> ## Push local branches to Github
> ## Remove local repository and do a fresh clone
> git clone https://github.com/myUsername/MyPackage.git
> cd MyPackage
> bash /path/to/update_remotes.sh
> git checkout devel
> git svn rebase
> git merge master --no-edit
> git svn dcommit --add-author-from
> git svn rebase
>
> ## For each conflict loop, do
> grep -lr '<<<<<<<' . | xargs git checkout --theirs
> git add *
> git rebase --continue
>
> ## Once that's done, do
> git svn dcommit --add-author-from
>
> ## Then pull from the Bioc mirror
> git pull
> ## Delete the master branch
> ## Make a new master branch which is a copy from the devel branch
>
> ## Continue as https://github.com/rmflight/categoryCompare is doing
>
>
>
> Cheers,
> Leo
>
> On Fri, Jun 19, 2015 at 2:27 PM, Dan Tenenbaum <dtenenba at fredhutch.org>
> wrote:
> >
> >
> > ----- Original Message -----
> >> From: "Guangchuang Yu" <gcyu at connect.hku.hk>
> >> To: bioc-devel at r-project.org
> >> Sent: Wednesday, June 17, 2015 5:04:04 AM
> >> Subject: Re: [Bioc-devel] Bioconductor Git/GitHub Mirrors
> >>
> >> Dear Dan,
> >>
> >> I update my package
> >> GOSemSim(https://github.com/GuangchuangYu/GOSemSim),
> >> with the following steps:
> >>
> >> 1. delete the git-svn-bridge
> >> 1. bash /path/to/update_remotes.sh
> >> 2. git checkout devel
> >> 3. git checkout master
> >> 4. git merge devel
> >>
> >> All was fine without any error.
> >>
> >
> > I asked Jim about this and he said this last step may have made git svn
> fail to find the svn information. You can do this command here:
> >
> > git reset --hard a3ff1e7a3efd09b0c
> >
> > That's the commit before the merge. You should then be able to run "git
> svn dcommit".
> >
> >
> > Dan
> >
> >
> >> But when I follow the instruction to test a commit to svn.
> >>
> >> 1. Each time you want to push git commits to svn:
> >> 1. git checkout devel to switch to the devel branch. (use
> >> release-X.X
> >> for release branches)
> >> 2. git svn rebase to get the latest SVN changes.
> >> 3. git merge master --no-edit to merge your changes from the
> >> master
> >> branch or skip this step and work directly on the current
> >> branch.
> >> 4. git svn rebase && git svn dcommit --add-author-from to sync
> >> and
> >> commit your changes to svn. You may be prompted here for your
> >> SVN username
> >> and password.
> >>
> >>
> >> ➜ GOSemSim git:(master) git checkout devel
> >> Switched to branch 'devel'
> >> Your branch is up-to-date with 'bioc/master'.
> >> ➜ GOSemSim git:(devel) git svn rebase
> >> Migrating from a git-svn v1 layout...
> >> Data from a previous version of git-svn exists, but
> >> .git/svn
> >> (required for this version (2.4.1) of git-svn) does not exist.
> >> Done migrating from a git-svn v1 layout
> >> Rebuilding
> >>
> .git/svn/refs/remotes/git-svn-devel/.rev_map.bc3139a8-67e5-0310-9ffc-ced21a209358
> >> ...
> >> r35719 = 714adc6875914b83b673bcf93f6eac976676388d
> >> r36178 = 199736e1f199d80a16121c0dca7e4aac015140bc
> >> ...
> >> ...
> >> r104302 = b7770b576acd1439d7e01a5115866fe6bb84d2d7
> >> r104400 = a3ff1e7a3efd09b0c0a760f2d4053331ee8039d2
> >> Done rebuilding
> >>
> .git/svn/refs/remotes/git-svn-devel/.rev_map.bc3139a8-67e5-0310-9ffc-ced21a209358
> >> Current branch devel is up to date.
> >>
> >> GOSemSim git:(devel) git merge master --no-edit
> >> Updating a3ff1e7..5b9b679
> >> Fast-forward
> >> ➜ GOSemSim git:(devel) git svn rebase && git svn dcommit
> >> --add-author-from
> >> Unable to determine upstream SVN information from working tree
> >> history
> >>
> >>
> >> It throw the above error. Any idea?
> >>
> >> Bests,
> >> Guangchuang
> >> --
> >> --~--~---------~--~----~------------~-------~--~----~
> >> Guangchuang Yu, PhD Candidate
> >> State Key Laboratory of Emerging Infectious Diseases
> >> School of Public Health
> >> The University of Hong Kong
> >> Hong Kong SAR, China
> >> www: http://ygc.name
> >> -~----------~----~----~----~------~----~------~--~---
> >>
> >> [[alternative HTML version deleted]]
> >>
> >> _______________________________________________
> >> Bioc-devel at r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >>
> >
> > _______________________________________________
> > Bioc-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
[[alternative HTML version deleted]]
More information about the Bioc-devel
mailing list