[Bioc-devel] Shouldn't we distinguish between package-specific and dependency errors?

Michael Lawrence lawrence.michael at gene.com
Thu Sep 24 15:07:12 CEST 2015


It would be good to know how these dependency breakages actually arise. Is
it the case that the culprit usually fails check (with a warning or
error?). It could just be a bug that gets by the tests in the dependency.
Those would be virtually impossible for the build system to figure out.

This underscores how one should select dependencies wisely, but as Wolfgang
points out, dependencies are often justified and highly desirable. If one
of your dependencies is causing a problem, reach out to the developers to
alert them to the issue and perhaps lend a helping hand.



On Thu, Sep 24, 2015 at 5:56 AM, Andrzej Oleś <andrzej.oles at gmail.com>
wrote:

> Hi,
>
> we need to distinguish here between build/install and check errors. The
> first ones hold the package update (instead, the last working version is
> used). On the other hand, check errors do not hold the package from
> propagating into the repository causing collateral damage (at least that's
> what I observe in the devel branch).
>
> A good example is EBImage which is currently broken for all architectures
> but Linux (see:
> http://bioconductor.org/checkResults/devel/bioc-LATEST/EBImage/). It
> doesn't affect it's downstream dependencies because the error occurs at
> build stage, see for example imageHTS (
> http://bioconductor.org/packages/3.2/bioc/html/imageHTS.html). Fair
> enough,
> EBImage has a red badge, whereas imageHTS has a green one.
>
> So the issue raised by Ludwig occurs only with packages which fail during
> check. Maybe changing the publication policy in such cases, i.e. hold the
> updated package from going into the repository when it fails 'R CMD check'
> would help to address the problem, at least for BioC packages?
>
> Best,
> Andrzej
>
> On Thu, Sep 24, 2015 at 2:22 PM, Wolfgang Huber <whuber at embl.de> wrote:
>
> > It seems that the “build” shield on the package landing page conflates
> > things that happen in the package, and in its dependencies.
> > Do we have a clear spec of what the purpose of that shield is?
> >
> > Something to avoid IMHO is creating incentives for package developers to
> > reduce dependencies to make their package “look" more robust, at the cost
> > of duplication or functionality.
> >
> > Wolfgang
> >
> >
> > > On 24 Sep 2015, at 14:13, Ludwig Geistlinger <
> > Ludwig.Geistlinger at bio.ifi.lmu.de> wrote:
> > >
> > >> Do you have any information on how often this kind of breakage occurs?
> > >
> > > Having my package ~1 year in, I would say that happened roughly 5 times
> > to
> > > me.
> > >
> > > I wonder whether other developers could comment on their experience
> with
> > > that as well.
> > >
> > >
> > >> On Thu, Sep 24, 2015 at 4:35 AM, Ludwig Geistlinger <
> > >> Ludwig.Geistlinger at bio.ifi.lmu.de> wrote:
> > >>
> > >>> Dear Bioc-Team,
> > >>>
> > >>> I would like to make this point brought up by Weijun more general.
> > >>> He reported a considerable number of packages to be broken by
> > >>> (recursively) depending on KEGGREST - which actually broke due to
> KEGG
> > >>> itself (however, this seems to be resolved by the current build).
> > >>>
> > >>> Nevertheless, given that a dependency can break your package at any
> > >>> time,
> > >>> it is currently hard to device a robust and stable software product
> > even
> > >>> within the semi-annual release.
> > >>>
> > >>
> > >> Do you have any information on how often this kind of breakage occurs?
> > >>
> > >>
> > >>> Thus, I wonder whether Bioc packages in release (at least those
> having
> > >>> other packages depending on them) shouldn't always be rolled back to
> > the
> > >>> last version that passed build and check without error, in order to
> > >>> ensure
> > >>> functioning of packages down the hierarchy.
> > >>>
> > >>> Based on these considerations, I also wonder whether the shield on
> the
> > >>> package landing page indicating the result of the package building
> > >>> (ok/warning/error) shouldn't distinguish between errors caused by
> > >>> dependencies and errors caused by the package itself.
> > >>>
> > >>> Imagine the not too unrealistic case of a new Bioc package presented
> in
> > >>> a
> > >>> Software article under review.
> > >>> Without doubt, a reviewer will be negatively influenced by the
> 'error'
> > >>> shield indicating that the package has not been properly worked out.
> > >>> This is fair enough if the package's own code produces these bugs,
> but
> > >>> the
> > >>> opposite it true if that is due to a broken dependency.
> > >>>
> > >>>
> > >> Recent developments at the Volkswagen company should help raise
> general
> > >> awareness that software development and maintenance is a fraught
> > process.
> > >> If
> > >> software S depends on software T and T is unreliable then so is S.
> The
> > >> negative influence of events of the sort you describe has potential
> > value.
> > >>
> > >> I believe there are ways of using containers so that software can be
> > >> distributed
> > >> in a verified working state, perhaps suitable for a fully predictable
> > >> review,
> > >> but I doubt this is a real solution to the actual problem.
> > >>
> > >>
> > >>
> > >>> In the worst case, the package will run fine the whole time the
> article
> > >>> is
> > >>> prepared, but breaks due to a broken dependency the day the reviewer
> > >>> starts to evaluate the manuscript.
> > >>>
> > >>> I know that this does not resolves problems of dependencies outside
> of
> > >>> BioC such as for KEGGREST with KEGG.
> > >>> But at least for dependencies within BioC, I wonder whether this is a
> > >>> point worth considering.
> > >>>
> > >>> Thanks & Best,
> > >>> Ludwig
> > >>>
> > >>>
> > >>> --
> > >>> Dipl.-Bioinf. Ludwig Geistlinger
> > >>>
> > >>> Lehr- und Forschungseinheit für Bioinformatik
> > >>> Institut für Informatik
> > >>> Ludwig-Maximilians-Universität München
> > >>> Amalienstrasse 17, 2. Stock, Büro A201
> > >>> 80333 München
> > >>>
> > >>> Tel.: 089-2180-4067
> > >>> eMail: Ludwig.Geistlinger at bio.ifi.lmu.de
> > >>>
> > >>>
> > >>>> Hi Weijun,
> > >>>>
> > >>>> ----- Original Message -----
> > >>>>> From: "Luo Weijun" <luo_weijun at yahoo.com>
> > >>>>> To: maintainer at bioconductor.org, dtenenba at fredhutch.org
> > >>>>> Cc: "Martin Morgan" <mtmorgan at fredhutch.org>,
> > >>> Bioc-devel at r-project.org
> > >>>>> Sent: Wednesday, September 23, 2015 9:44:13 AM
> > >>>>> Subject: KEGG REST issue
> > >>>>>
> > >>>>> Dear BioC team,
> > >>>>> I noticed some problem with keggLink() function of KEGGREST
> package,
> > >>>>> and it can be traced back to KEGG REST API Linked entries. Some of
> > >>>>> this API function is broken. For example, the following line used
> to
> > >>>>> get all gene-pathway mapping for human, but retrieves nothing now.
> > >>>>> path.hsa= KEGGREST::keggLink("pathway", "hsa")
> > >>>>>
> > >>>>> In fact, these two bulk queries with the rest api don’t work
> > >>> anymore.
> > >>>>> http://rest.kegg.jp/link/pathway/hsa
> > >>>>> http://rest.kegg.jp/link/hsa/pathway
> > >>>>> but smaller queries on Linked entries seem to be fine. not sure
> > >>>>> whether other REST API functions are affected or not. As a
> > >>>>> consequence, KEGGREST and many dependent packages had build error.
> > >>>>>
> > >>>
> >
> http://bioconductor.org/checkResults/release/bioc-LATEST/KEGGREST/morelia-buildsrc.html
> > >>>>>
> > >>>>> anway, just want you know about this, see if you can do anything on
> > >>>>> this.
> > >>>>
> > >>>> Yes, I am aware of this. It's an issue on the KEGG side and I have
> > >>>> contacted the KEGG team. I have not heard back yet.
> > >>>>
> > >>>> Dan
> > >>>>
> > >>>>
> > >>>>> Weijun
> > >>>>>
> > >>>>>
> > >>>>
> > >>>> _______________________________________________
> > >>>> Bioc-devel at r-project.org mailing list
> > >>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> > >>>>
> > >>>
> > >>> _______________________________________________
> > >>> Bioc-devel at r-project.org mailing list
> > >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> > >>>
> > >>
> > >
> > >
> > > --
> > > Dipl.-Bioinf. Ludwig Geistlinger
> > >
> > > Lehr- und Forschungseinheit für Bioinformatik
> > > Institut für Informatik
> > > Ludwig-Maximilians-Universität München
> > > Amalienstrasse 17, 2. Stock, Büro A201
> > > 80333 München
> > >
> > > Tel.: 089-2180-4067
> > > eMail: Ludwig.Geistlinger at bio.ifi.lmu.de
> > >
> > > _______________________________________________
> > > Bioc-devel at r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >
> > _______________________________________________
> > Bioc-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

	[[alternative HTML version deleted]]



More information about the Bioc-devel mailing list