[Bioc-devel] Use and Usability metrics / shields

Dan Tenenbaum dtenenba at fredhutch.org
Tue May 19 21:47:02 CEST 2015 


----- Original Message -----
> From: "Leonardo Collado Torres" <lcollado at jhu.edu>
> To: "Dan Tenenbaum" <dtenenba at fredhutch.org>
> Cc: "Jim Hester" <james.f.hester at gmail.com>, bioc-devel at r-project.org
> Sent: Tuesday, May 19, 2015 12:37:18 PM
> Subject: Re: [Bioc-devel] Use and Usability metrics / shields
> 
> Regarding the 'posts' tag, I can see that it includes a "closed
> questions" component. For example,
> http://www.bioconductor.org/packages/release/bioc/html/derfinder.html
> is 3/1/9/0 right now meaning that 0 questions are closed. From
> https://support.bioconductor.org/info/faq/, only moderators can close
> questions. That seems like quite a bit of work for the moderators. So
> maybe it would be best to drop the "closed questions" component. Or
> alternatively, can the author of a package moderate the posts that
> have a tag corresponding to their package?
> 

Perhaps the wording is wrong; what 'closed' is supposed to mean is that the original poster has accepted an answer. I'll change 'closed' to 'accepted'. 


> As for 'build: warnings', it seems like it will show for some devel
> packages all the time. For example,
> http://www.bioconductor.org/packages/devel/bioc/html/regionReport.html
> always has a warning in the Windows build machine due to a mismatch
> in
> the version of Rtools installed.

This is a bug in devtools and may have already been fixed (but not yet propagated to CRAN).
IMO this should be reflected in the build shield.

Dan


> 
> I do like these changes and the addition of shields =)
> 
> 
> On Thu, May 14, 2015 at 10:56 AM, Dan Tenenbaum
> <dtenenba at fredhutch.org> wrote:
> >
> > ----- Original Message -----
> >> From: "Jim Hester" <james.f.hester at gmail.com>
> >> To: "Martin Morgan" <mtmorgan at fredhutch.org>
> >> Cc: bioc-devel at r-project.org
> >> Sent: Thursday, May 14, 2015 7:53:03 AM
> >> Subject: Re: [Bioc-devel] Use and Usability metrics / shields
> >>
> >> The common shield convention is to use blue or orange when the
> >> information
> >> is not qualitatively good or bad, but the color choice is just
> >> subjective
> >> in the end.
> >
> > It does seem though that we should indicate the non-changing nature
> > of these shields with some kind of color change. Perhaps we can
> > come up with one that works with the other design elements on the
> > page.
> >
> > BTW, the 'posts' tag does change color; if there are 0 posts tagged
> > with a package name, the shield is yellow; otherwise it's green.
> >
> > Dan
> >
> >
> >>
> >> On Thu, May 14, 2015 at 10:47 AM, Martin Morgan
> >> <mtmorgan at fredhutch.org>
> >> wrote:
> >>
> >> > On 05/10/2015 11:39 AM, COMMO Frederic wrote:
> >> >
> >> >> Dear Martin,
> >> >>
> >> >> All of these suggestions sound good.
> >> >>
> >> >> Wolfgang's suggestion regarding possible associated papers
> >> >> might
> >> >> be also
> >> >> great.
> >> >>
> >> >> Another useful information would be to point to other
> >> >> publications
> >> >> where
> >> >> a given package was used, and cited.
> >> >> I don't know if it's technically possible, but it would be
> >> >> greatly
> >> >> informative to know how frequently a package is used, and how
> >> >> it
> >> >> performs,
> >> >> in real contexts.
> >> >>
> >> >> Frederic Commo
> >> >> Bioinformatics, U981
> >> >> Gustave Roussy
> >> >>
> >> >> ________________________________________
> >> >> De : Bioc-devel [bioc-devel-bounces at r-project.org] de la part
> >> >> de
> >> >> Wolfgang Huber [whuber at embl.de]
> >> >> Date d'envoi : samedi 9 mai 2015 19:57
> >> >> À : Martin Morgan
> >> >> Cc: bioc-devel at r-project.org
> >> >> Objet : Re: [Bioc-devel] Use and Usability metrics / shields
> >> >>
> >> >> Dear Martin
> >> >>
> >> >> great idea.
> >> >> "Current build status” could perhaps be wrapped with
> >> >> "Cross-platform
> >> >> availability” into some sort of “Availability / Accessibility”?
> >> >>
> >> >> I wonder how informative it would be to make metrics such as
> >> >> (i) citations of the associated paper
> >> >> (ii) full-text mentions e.g. in PubmedCentral
> >> >> actually useful. (i) could be flawed if package and paper are
> >> >> diverged;
> >> >> (ii) would require good disambiguation, e.g. like bioNerDS
> >> >> http://www.biomedcentral.com/1471-2105/14/194 (or other tools?
> >> >> not
> >> >> my
> >> >> expertise). Do we have someone with capabilities in this area
> >> >> on
> >> >> this list?
> >> >>
> >> >>
> >> > Thanks for these suggestions.
> >> >
> >> > I like the idea of linking into the scientific literature,
> >> > initially as
> >> > part of the 'Citation' section on each landing page rather than
> >> > as
> >> > a shield
> >> > (maybe a shield in the long term). As Wolfgang mentions it is a
> >> > little more
> >> > challenging than a one-liner to match the information available
> >> > from a
> >> > CITATION file (or automatically generated) to an appropriate
> >> > search
> >> > in
> >> > PubMed, and because citations are an important formal metric it
> >> > seems
> >> > important to get this more-or-less right.
> >> >
> >> > For what it's worth the more-or-less continuous stream of papers
> >> > citing
> >> > 'Biocondcutor' are listed at
> >> >
> >> >   http://bioconductor.org/help/publications/
> >> >
> >> > The links in the 'Literature Search' box query various resources
> >> > for use
> >> > of the term 'Bioconductor'.
> >> >
> >> > We have so far kept the distinction between 'available' and
> >> > 'build',
> >> > partly because builds sometimes fail for transient (e.g.,
> >> > connectivity)
> >> > reasons or, in devel, because of an incomplete check-in that
> >> > does
> >> > not
> >> > compromise the end-user availability and functionality of the
> >> > version
> >> > available via biocLite().
> >> >
> >> > It's kind of amusing that (a) most of the information was
> >> > already
> >> > available, often on the landing page (like the links to build
> >> > reports that
> >> > Henrik mentions, or years in bioc), so the shields are serving
> >> > just
> >> > to
> >> > emphasize these; and (b) the 'green' implies some-how 'good',
> >> > but
> >> > many of
> >> > the shields (e.g., years in Bioc, posts, commits, downloads) are
> >> > actually
> >> > never not green. Maybe these shields should be white?
> >> >
> >> > Thanks again for the feedback; initial response seems to be
> >> > positive.
> >> >
> >> > Martin
> >> >
> >> >
> >> >  PS  Martin you’ll like Fig. 2 of their paper.
> >> >>
> >> >> Wolfgang
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>  On May 9, 2015, at 19:15 GMT+2, Martin Morgan
> >> >>  <mtmorgan at fredhutch.org>
> >> >>> wrote:
> >> >>>
> >> >>> Bioc developers!
> >> >>>
> >> >>> It's important that our users be able to identify packages
> >> >>> that
> >> >>> are
> >> >>> suitable for their research question. Obviously a first step
> >> >>> is
> >> >>> to identify
> >> >>> packages in the appropriate research domain, for instance
> >> >>> through
> >> >>> biocViews.
> >> >>>
> >> >>>   http://bioconductor.org/packages/release/
> >> >>>
> >> >>> We'd like to help users further prioritize their efforts by
> >> >>> summarizing
> >> >>> use and usability. Metrics include:
> >> >>>
> >> >>> - Cross-platform availability -- biocLite()-able from all or
> >> >>> only
> >> >>> some
> >> >>> platforms
> >> >>> - Support forum activity -- questions and comments /
> >> >>> responses, 6
> >> >>> month
> >> >>> window
> >> >>> - Download percentile -- top 5, 20, 50%, or 'available'
> >> >>> - Current build status -- errors or warnings on some or all
> >> >>> platforms
> >> >>> - Developer activity -- commits in the last 6 months
> >> >>> - Historical presence -- years in Bioconductor
> >> >>>
> >> >>> Obviously the metrics are imperfect, so constructive feedback
> >> >>> welcome --
> >> >>> we think the above capture in a more-or-less objective and
> >> >>> computable way
> >> >>> the major axes influencing use and usability.
> >> >>>
> >> >>> We initially intend to prominently display 'shields' (small
> >> >>> graphical
> >> >>> icons) on package landing pages.
> >> >>>
> >> >>> Thanks in advance for your comments,
> >> >>>
> >> >>> Martin Morgan
> >> >>> Bioconductor
> >> >>> --
> >> >>> Computational Biology / Fred Hutchinson Cancer Research Center
> >> >>> 1100 Fairview Ave. N.
> >> >>> PO Box 19024 Seattle, WA 98109
> >> >>>
> >> >>> Location: Arnold Building M1 B861
> >> >>> Phone: (206) 667-2793
> >> >>>
> >> >>> _______________________________________________
> >> >>> Bioc-devel at r-project.org mailing list
> >> >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >> >>>
> >> >>
> >> >> _______________________________________________
> >> >> Bioc-devel at r-project.org mailing list
> >> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >> >>
> >> >>
> >> >
> >> > --
> >> > Computational Biology / Fred Hutchinson Cancer Research Center
> >> > 1100 Fairview Ave. N.
> >> > PO Box 19024 Seattle, WA 98109
> >> >
> >> > Location: Arnold Building M1 B861
> >> > Phone: (206) 667-2793
> >> >
> >> > _______________________________________________
> >> > Bioc-devel at r-project.org mailing list
> >> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >> >
> >>
> >>       [[alternative HTML version deleted]]
> >>
> >> _______________________________________________
> >> Bioc-devel at r-project.org mailing list
> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> >>
> >
> > _______________________________________________
> > Bioc-devel at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

More information about the Bioc-devel mailing list