[Bioc-devel] Duplicate commit error on and off

Turaga, Nitesh Nitesh.Turaga at RoswellPark.org
Fri Oct 6 14:40:10 CEST 2017 

This is good news!


> On Oct 5, 2017, at 7:06 PM, Thomas Girke <thomas.girke at ucr.edu> wrote:
> 
> Thanks. I have synced everything to the current development and was able to commit back to Bioc-upstream/master.
> 
> Thomas
> 
> On Thu, Oct 5, 2017 at 10:28 AM Turaga, Nitesh <Nitesh.Turaga at roswellpark.org> wrote:
> Done. If you clone from Bioconductor again, you should get a clean copy just as after the transition.
> 
> If you have questions let me know.
> 
> Best,
> 
> Nitesh
> 
> 
> > On Oct 5, 2017, at 1:05 PM, Thomas Girke <thomas.girke at ucr.edu> wrote:
> >
> > Yes, let's just do it this way.
> >
> > Thomas
> >
> > On Thu, Oct 5, 2017 at 9:36 AM Turaga, Nitesh <Nitesh.Turaga at roswellpark.org> wrote:
> > It would be easier. I can do it and it would take you back to,
> >
> > commit c84c7b5fbdf1419af5030b66f8a759d29307f40b (git-svn)
> > Author: Herve Pages <hpages at fhcrc.org>
> > Date:   Mon Apr 24 19:50:57 2017 +0000
> >
> >     bump x.y.z versions to odd y after creation of 3_5 branch
> >
> >     git-svn-id: file:///home/git/hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/systemPipeR@129129 bc3139a8-67e5-0310-9ffc-ced21a209358
> >
> >
> > Give me a thumbs up and I’ll go ahead.
> >
> > > On Oct 5, 2017, at 11:03 AM, Thomas Girke <thomas.girke at ucr.edu> wrote:
> > >
> > > Hi Nitesh,
> > >
> > > If the reset to #c84c7b5f is the best option then let's do this.
> > >
> > > Just wondering, wouldn't it be much easier to run on your end
> > >
> > > git reset --hard <<Commit_ID_prior_to_problematic_commit>>
> > > git merge --squash HEAD@{1}
> > >
> > > This would deduplicate everything without losing any commits.
> > >
> > > Thanks,
> > >
> > > Thomas
> > >
> > > On Thu, Oct 05, 2017 at 10:56:53AM +0000, Turaga, Nitesh wrote:
> > >> Hi Thomas,
> > >>
> > >> Sorry for the email sent to the wrong place earlier.
> > >>
> > >> One way I can help you is, that I can take you back to the “clean” state on August 16th to commit (c84c7b5fbdf1419af5030b66f8a759d29307f40b).
> > >>
> > >> i.e before the commit
> > >>
> > >> commit 500a8b924b7c8bfd9b4e960b471e1e000032c06f
> > >> Merge: e6ff3f8 3cd5aee
> > >> Author: tgirke <tgirke at citrus.ucr.edu>
> > >> Date:   Thu Aug 17 18:55:36 2017 -0700
> > >>
> > >>    Fixed conflicts in version change
> > >>
> > >> You would have to manually replay all the commits after that on top of that. By this, I mean, all the commits after 500a8b9, need to be cherry-picked and pushed to the repo. But you need to be careful here, because only “non-merge” commits should be cherry picked. This avoids the duplicates on your contaminated branch to not enter the new clean “master” branch.
> > >>
> > >> Non merge commits,
> > >> commit 75bca921c1e2c85e8ab36b5a2d006fb2837c1c50
> > >> commit 49fa6d434c7bc690223c2b0c88c5353386eb39ba
> > >> commit c73fb18ad737bea84b5c7bc3967c9bea8bcd4b9c
> > >>
> > >>
> > >> After doing that, your last 4 commits will look like this,
> > >>
> > >>
> > >> commit 9fc03a5719486e333bc1182b07cd6c5e15e8798d (HEAD -> master)
> > >> Author: Hervé Pagès <hpages at fredhutch.org>
> > >> Date:   Thu Aug 31 22:28:06 2017 -0700
> > >>
> > >>    restore empty folder lost during svn-to-git transition
> > >>
> > >> commit 2edc489e4d3ca642a240f062c34528e0d918e11e
> > >> Author: tgirke <tgirke at citrus.ucr.edu>
> > >> Date:   Wed Aug 23 19:21:04 2017 -0700
> > >>
> > >>    moduleload with serveral envir variables
> > >>
> > >> commit 1d05c70f27ca083d76bca7ef7be59f43662ba07c
> > >> Author: tgirke <tgirke at citrus.ucr.edu>
> > >> Date:   Thu Aug 17 20:29:38 2017 -0700
> > >>
> > >>    Fixed gene set naming problem in GOCluster_Report
> > >>
> > >> commit c84c7b5fbdf1419af5030b66f8a759d29307f40b (git-svn)
> > >> Author: Herve Pages <hpages at fhcrc.org>
> > >> Date:   Mon Apr 24 19:50:57 2017 +0000
> > >>
> > >>    bump x.y.z versions to odd y after creation of 3_5 branch
> > >>
> > >>    git-svn-id: file:///home/git/hedgehog.fhcrc.org/bioconductor/trunk/madman/Rpacks/systemPipeR@129129 bc3139a8-67e5-0310-9ffc-ced21a209358
> > >>
> > >>
> > >> This has a couple of consequences,
> > >>
> > >> 1. you need to follow (http://master.bioconductor.org/developers/how-to/git/abandon-changes/#force-bioconductor--to-github- ) to switch you master branch to follow bioconductor exactly.
> > >>
> > >> 2. you and your collaborators, need to re-clone from your GitHub, and work on the new master branch abandoning the old one.
> > >>
> > >> 3. Make sure that all the file changes you’ve made have been moved over.
> > >>
> > >> Let me know if this works for you, and we can go ahead. Another option is, I can do this for you, with a clean “master” on the Bioconductor repo, and you can follow the “force bioconductor-to-github” manual on your end.
> > >>
> > >> Nitesh
> > >>
> > >>
> > >>> On Oct 4, 2017, at 2:38 PM, Thomas Girke <thomas.girke at ucr.edu> wrote:
> > >>>
> > >>> I didn't commit the duplicate fix with git merge --squash since I wanted to do the branch swap as I did with ChemmineR, but now this is not working anymore or there are other problems?
> > >>>
> > >>> One possible solution could be to allow me a one time push with the -f flag with the deduplicated master branch. According to the instructions here (https://goo.gl/xnr9j8) this should be possible but I think it has been disabled. Some of the Bioc/FAQs related to this topic indicate this too.
> > >>>
> > >>> Thomas
> > >>>
> > >>> On Wed, Oct 4, 2017 at 11:27 AM Turaga, Nitesh <Nitesh.Turaga at roswellpark.org> wrote:
> > >>> Thanks, I’ll get back to you. I wasn’t sure because the history on that is not the same. It is a few commits short.
> > >>>
> > >>>
> > >>>
> > >>>> On Oct 4, 2017, at 2:00 PM, Thomas Girke <thomas.girke at ucr.edu> wrote:
> > >>>>
> > >>>> The gitub repos is here:
> > >>>> https://github.com/tgirke/systemPipeR
> > >>>> It is also linked from the Bioc page of the package here: http://bioconductor.org/packages/systemPipeR/
> > >>>>
> > >>>> Thomas
> > >>>>
> > >>>> On Wed, Oct 4, 2017 at 8:11 AM Turaga, Nitesh <Nitesh.Turaga at roswellpark.org> wrote:
> > >>>> Where is your primary development repo on Github? Please send me link.
> > >>>>
> > >>>> Best,
> > >>>>
> > >>>> Nitesh
> > >>>>
> > >>>>> On Oct 4, 2017, at 10:14 AM, Turaga, Nitesh <Nitesh.Turaga at RoswellPark.org> wrote:
> > >>>>>
> > >>>>> Hi Thomas,
> > >>>>>
> > >>>>> The following issue occurred because there was a commit on Aug 17th with all the duplicates. Unfortunately, your upstream repository has been contaminated with duplicate commits.
> > >>>>>
> > >>>>> Till I figure out a solution on how to fix this, please hold off any further commits.
> > >>>>>
> > >>>>> I’ll keep you posted. You might have to follow a few special instructions to add any commits you have on your local machine now.
> > >>>>>
> > >>>>> Best,
> > >>>>>
> > >>>>> Nitesh
> > >>>>>
> > >>>>>> On Oct 3, 2017, at 10:08 PM, Thomas Girke <thomas.girke at ucr.edu> wrote:
> > >>>>>>
> > >>>>>> Hi Nitesh,
> > >>>>>>
> > >>>>>> Sorry for bothering you again about a similar problem. The master/devel branch of
> > >>>>>> my systemPipeR package also contains upstream duplicates in the Bioc repos. However, I am
> > >>>>>> not able to resolve the problem entirely. When using the 'git merge --squash' approach,
> > >>>>>> that worked before, then the duplicates can be removed but this time I am ending up in
> > >>>>>> a trap at the push step:
> > >>>>>>
> > >>>>>> git push upstream master
> > >>>>>>
> > >>>>>> where I am getting the following error:
> > >>>>>>
> > >>>>>> To git at git.bioconductor.org:packages/systemPipeR.git
> > >>>>>> ! [rejected]        master -> master (non-fast-forward)
> > >>>>>> error: failed to push some refs to 'git at git.bioconductor.org:packages/systemPipeR.git'
> > >>>>>> hint: Updates were rejected because the tip of your current branch is behind
> > >>>>>> hint: its remote counterpart. Integrate the remote changes (e.g.
> > >>>>>> hint: 'git pull ...') before pushing again.
> > >>>>>> hint: See the 'Note about fast-forwards' in 'git push --help' for details.
> > >>>>>>
> > >>>>>> The solution to this would normally be to run a `git pull` first but when I do this
> > >>>>>> then I am getting all the duplicates back which puts me back where I started.
> > >>>>>>
> > >>>>>> Running a git push with the -f argument also doesn't work since the usage of this option
> > >>>>>> gets rejected by the remote.
> > >>>>>>
> > >>>>>> Do you have any suggestions what else I should try?
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Thomas
> > >>>>>>
> > >>>>>>
> > >>>>>> On Tue, Sep 12, 2017 at 02:19:44AM +0000, Thomas Girke wrote:
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> The following allowed me to eliminate the duplicated commits in one step
> > >>>>>>> via git merge --squash and then successfully push back to the bioc-git
> > >>>>>>> server. After this I was able to switch to the swap branch approach to
> > >>>>>>> avoid similar problems in the future.
> > >>>>>>>
> > >>>>>>> Example here for master branch:
> > >>>>>>>
> > >>>>>>> git checkout master
> > >>>>>>> git pull upstream master # just in case
> > >>>>>>> git reset --hard <commit_id> # Reset the current branch to the
> > >>>>>>> commit right before dups started
> > >>>>>>> git merge --squash HEAD@{1} # Squashes duplicated commits from
> > >>>>>>> chosen <commit_id> to HEAD@{1} (state right before previous reset
> > >>>>>>> step)
> > >>>>>>> git commit -am "some_message" # Commit squashed changes
> > >>>>>>> git push upstream master # Push to bioc-git server
> > >>>>>>>
> > >>>>>>> I am not sure if the above is the best solution but I thought I report it
> > >>>>>>> here in case others experience similar problems. BTW: in my case the
> > >>>>>>> duplicates were all generated in the upstream merge (step 6) of the
> > >>>>>>> instructions here: https://goo.gl/wWVEeT. None of the parent branches (on
> > >>>>>>> github or bioc) used in this merge step contained duplicated commits at
> > >>>>>>> least as far as I have checked so far. Perhaps some of this relates back to
> > >>>>>>> the git svn/rebase steps we used under the old git mirror?
> > >>>>>>>
> > >>>>>>> Just in case, the following command is very helpful to identify duplicate
> > >>>>>>> commits based on patch-id. Commits with identical patch-ids are very likely
> > >>>>>>> to have identical content.
> > >>>>>>>
> > >>>>>>> git rev-list master | xargs -r -L1 git diff-tree -m -p | git
> > >>>>>>> patch-id | sort | uniq -w40 -D | cut -c42-80  | xargs -r git log
> > >>>>>>> --no-walk --pretty=format:"%h %ad %an (%cn) %s" --date-order
> > >>>>>>> --date=iso
> > >>>>>>>
> > >>>>>>> After duplicated commit pairs have been identified, one can check with diff
> > >>>>>>> or vimdiff whether their content is identical:
> > >>>>>>>
> > >>>>>>> git --no-pager show <commit_id1> > zzz1
> > >>>>>>> git --no-pager show <commit_id2> > zzz2
> > >>>>>>> vimdiff zzz1 zzz2
> > >>>>>>>
> > >>>>>>> Thomas
> > >>>>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
> > >>>>> _______________________________________________
> > >>>>> Bioc-devel at r-project.org mailing list
> > >>>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> > >>>>
> > >>>>
> > >>>>
> > >>>> This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
> > >>>
> > >>>
> > >>>
> > >>> This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
> > >>
> > >>
> > >>
> > >> This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
> >
> >
> >
> > This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.
> 
> 
> 
> This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.



This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.

More information about the Bioc-devel mailing list