[Bioc-devel] Use and Usability metrics / shields

[Leonardo Collado Torres]

Ahh, I missed the 2 versions on the "platforms" shield. I am using
both "build" shields. I'll make some edits then.

On Fri, Jun 12, 2015 at 2:49 PM, Dan Tenenbaum <dtenenba at fredhutch.org> wrote:
>
>
> ----- Original Message -----
>> From: "Leonardo Collado Torres" <lcollado at jhu.edu>
>> To: "Dan Tenenbaum" <dtenenba at fredhutch.org>
>> Cc: bioc-devel at r-project.org
>> Sent: Friday, June 12, 2015 11:43:09 AM
>> Subject: Re: [Bioc-devel] Use and Usability metrics / shields
>>
>> Awesome!
>>
>> Here's the code in case others want to add the shields to their
>> github
>> readme files.
>>  https://gist.github.com/lcolladotor/de2e0b67fbf33518b922
>>
>>
>> derfinder example (software package)
>> https://github.com/lcolladotor/derfinder
>>
>> derfinderData example (experiment data package)
>> https://github.com/leekgroup/derfinderData
>>
>>
>
> Cool! Note that some shields will be the same for release and devel; others will differ, such as the "platforms" shield.
> So since your repos is presumably mapped to the devel version of your package, you should pull the 'platforms' shield from 'shields/availability/devel' instead of 'shields/availability/release'.
>
> Dan
>
>
>
>
>
>> On Fri, Jun 12, 2015 at 12:32 AM, Dan Tenenbaum
>> <dtenenba at fredhutch.org> wrote:
>> >
>> >
>> > ----- Original Message -----
>> >> From: "Leonardo Collado Torres" <lcollado at jhu.edu>
>> >> To: "Dan Tenenbaum" <dtenenba at fredhutch.org>
>> >> Cc: bioc-devel at r-project.org
>> >> Sent: Thursday, June 11, 2015 9:26:13 PM
>> >> Subject: Re: [Bioc-devel] Use and Usability metrics / shields
>> >>
>> >> Is it ok if we use the Bioconductor shields elsewhere? For
>> >> example,
>> >> in
>> >> a github repo landing page or in our website listing the software
>> >> we've contributed to.
>> >>
>> >> In particular, I'm thinking of adding
>> >> http://www.bioconductor.org/shields/posts/derfinder.svg (and the
>> >> other
>> >> shields) to https://github.com/lcolladotor/derfinder to go along
>> >> the
>> >> Travis CI shield I use right now.
>> >>
>> >
>> > Feel free, that was part of the idea.
>> >
>> > Dan
>> >
>> >
>> >> On Thu, Jun 4, 2015 at 11:22 AM, Jim Hester
>> >> <james.f.hester at gmail.com> wrote:
>> >> > Henrik,
>> >> >
>> >> > While I proposed the idea for the shields/badges Dan gets all
>> >> > the
>> >> > credit
>> >> > for the implementation.
>> >> >
>> >> > As far as your (implied) idea of a coverage badge, the thought
>> >> > had
>> >> > occurred
>> >> > to us!
>> >> >
>> >> > Jim
>> >> >
>> >> > On Wed, May 20, 2015 at 1:34 PM, Henrik Bengtsson
>> >> > <henrik.bengtsson at ucsf.edu
>> >> >> wrote:
>> >> >
>> >> >> So, lots of things are happening in a few months: Jim Hester
>> >> >> starts
>> >> >> working at Bioconductor, we get Bioc shields/badges, Jim's covr
>> >> >> package is released on CRAN, snare drum, ...  am I to eager if
>> >> >> I
>> >> >> already now start wishing for a hi-hat as well?
>> >> >>
>> >> >> /Henrik
>> >> >>
>> >> >> On Tue, May 19, 2015 at 12:47 PM, Dan Tenenbaum
>> >> >> <dtenenba at fredhutch.org>
>> >> >> wrote:
>> >> >> >
>> >> >> >
>> >> >> > ----- Original Message -----
>> >> >> >> From: "Leonardo Collado Torres" <lcollado at jhu.edu>
>> >> >> >> To: "Dan Tenenbaum" <dtenenba at fredhutch.org>
>> >> >> >> Cc: "Jim Hester" <james.f.hester at gmail.com>,
>> >> >> >> bioc-devel at r-project.org
>> >> >> >> Sent: Tuesday, May 19, 2015 12:37:18 PM
>> >> >> >> Subject: Re: [Bioc-devel] Use and Usability metrics /
>> >> >> >> shields
>> >> >> >>
>> >> >> >> Regarding the 'posts' tag, I can see that it includes a
>> >> >> >> "closed
>> >> >> >> questions" component. For example,
>> >> >> >> http://www.bioconductor.org/packages/release/bioc/html/derfinder.html
>> >> >> >> is 3/1/9/0 right now meaning that 0 questions are closed.
>> >> >> >> From
>> >> >> >> https://support.bioconductor.org/info/faq/, only moderators
>> >> >> >> can
>> >> >> >> close
>> >> >> >> questions. That seems like quite a bit of work for the
>> >> >> >> moderators. So
>> >> >> >> maybe it would be best to drop the "closed questions"
>> >> >> >> component. Or
>> >> >> >> alternatively, can the author of a package moderate the
>> >> >> >> posts
>> >> >> >> that
>> >> >> >> have a tag corresponding to their package?
>> >> >> >>
>> >> >> >
>> >> >> > Perhaps the wording is wrong; what 'closed' is supposed to
>> >> >> > mean
>> >> >> > is that
>> >> >> the original poster has accepted an answer. I'll change
>> >> >> 'closed'
>> >> >> to
>> >> >> 'accepted'.
>> >> >> >
>> >> >> >
>> >> >> >> As for 'build: warnings', it seems like it will show for
>> >> >> >> some
>> >> >> >> devel
>> >> >> >> packages all the time. For example,
>> >> >> >> http://www.bioconductor.org/packages/devel/bioc/html/regionReport.html
>> >> >> >> always has a warning in the Windows build machine due to a
>> >> >> >> mismatch
>> >> >> >> in
>> >> >> >> the version of Rtools installed.
>> >> >> >
>> >> >> > This is a bug in devtools and may have already been fixed
>> >> >> > (but
>> >> >> > not yet
>> >> >> propagated to CRAN).
>> >> >> > IMO this should be reflected in the build shield.
>> >> >> >
>> >> >> > Dan
>> >> >> >
>> >> >> >
>> >> >> >>
>> >> >> >> I do like these changes and the addition of shields =)
>> >> >> >>
>> >> >> >>
>> >> >> >> On Thu, May 14, 2015 at 10:56 AM, Dan Tenenbaum
>> >> >> >> <dtenenba at fredhutch.org> wrote:
>> >> >> >> >
>> >> >> >> > ----- Original Message -----
>> >> >> >> >> From: "Jim Hester" <james.f.hester at gmail.com>
>> >> >> >> >> To: "Martin Morgan" <mtmorgan at fredhutch.org>
>> >> >> >> >> Cc: bioc-devel at r-project.org
>> >> >> >> >> Sent: Thursday, May 14, 2015 7:53:03 AM
>> >> >> >> >> Subject: Re: [Bioc-devel] Use and Usability metrics /
>> >> >> >> >> shields
>> >> >> >> >>
>> >> >> >> >> The common shield convention is to use blue or orange
>> >> >> >> >> when
>> >> >> >> >> the
>> >> >> >> >> information
>> >> >> >> >> is not qualitatively good or bad, but the color choice is
>> >> >> >> >> just
>> >> >> >> >> subjective
>> >> >> >> >> in the end.
>> >> >> >> >
>> >> >> >> > It does seem though that we should indicate the
>> >> >> >> > non-changing
>> >> >> >> > nature
>> >> >> >> > of these shields with some kind of color change. Perhaps
>> >> >> >> > we
>> >> >> >> > can
>> >> >> >> > come up with one that works with the other design elements
>> >> >> >> > on
>> >> >> >> > the
>> >> >> >> > page.
>> >> >> >> >
>> >> >> >> > BTW, the 'posts' tag does change color; if there are 0
>> >> >> >> > posts
>> >> >> >> > tagged
>> >> >> >> > with a package name, the shield is yellow; otherwise it's
>> >> >> >> > green.
>> >> >> >> >
>> >> >> >> > Dan
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >>
>> >> >> >> >> On Thu, May 14, 2015 at 10:47 AM, Martin Morgan
>> >> >> >> >> <mtmorgan at fredhutch.org>
>> >> >> >> >> wrote:
>> >> >> >> >>
>> >> >> >> >> > On 05/10/2015 11:39 AM, COMMO Frederic wrote:
>> >> >> >> >> >
>> >> >> >> >> >> Dear Martin,
>> >> >> >> >> >>
>> >> >> >> >> >> All of these suggestions sound good.
>> >> >> >> >> >>
>> >> >> >> >> >> Wolfgang's suggestion regarding possible associated
>> >> >> >> >> >> papers
>> >> >> >> >> >> might
>> >> >> >> >> >> be also
>> >> >> >> >> >> great.
>> >> >> >> >> >>
>> >> >> >> >> >> Another useful information would be to point to other
>> >> >> >> >> >> publications
>> >> >> >> >> >> where
>> >> >> >> >> >> a given package was used, and cited.
>> >> >> >> >> >> I don't know if it's technically possible, but it
>> >> >> >> >> >> would
>> >> >> >> >> >> be
>> >> >> >> >> >> greatly
>> >> >> >> >> >> informative to know how frequently a package is used,
>> >> >> >> >> >> and
>> >> >> >> >> >> how
>> >> >> >> >> >> it
>> >> >> >> >> >> performs,
>> >> >> >> >> >> in real contexts.
>> >> >> >> >> >>
>> >> >> >> >> >> Frederic Commo
>> >> >> >> >> >> Bioinformatics, U981
>> >> >> >> >> >> Gustave Roussy
>> >> >> >> >> >>
>> >> >> >> >> >> ________________________________________
>> >> >> >> >> >> De : Bioc-devel [bioc-devel-bounces at r-project.org] de
>> >> >> >> >> >> la
>> >> >> >> >> >> part
>> >> >> >> >> >> de
>> >> >> >> >> >> Wolfgang Huber [whuber at embl.de]
>> >> >> >> >> >> Date d'envoi : samedi 9 mai 2015 19:57
>> >> >> >> >> >> À : Martin Morgan
>> >> >> >> >> >> Cc: bioc-devel at r-project.org
>> >> >> >> >> >> Objet : Re: [Bioc-devel] Use and Usability metrics /
>> >> >> >> >> >> shields
>> >> >> >> >> >>
>> >> >> >> >> >> Dear Martin
>> >> >> >> >> >>
>> >> >> >> >> >> great idea.
>> >> >> >> >> >> "Current build status” could perhaps be wrapped with
>> >> >> >> >> >> "Cross-platform
>> >> >> >> >> >> availability” into some sort of “Availability /
>> >> >> >> >> >> Accessibility”?
>> >> >> >> >> >>
>> >> >> >> >> >> I wonder how informative it would be to make metrics
>> >> >> >> >> >> such
>> >> >> >> >> >> as
>> >> >> >> >> >> (i) citations of the associated paper
>> >> >> >> >> >> (ii) full-text mentions e.g. in PubmedCentral
>> >> >> >> >> >> actually useful. (i) could be flawed if package and
>> >> >> >> >> >> paper
>> >> >> >> >> >> are
>> >> >> >> >> >> diverged;
>> >> >> >> >> >> (ii) would require good disambiguation, e.g. like
>> >> >> >> >> >> bioNerDS
>> >> >> >> >> >> http://www.biomedcentral.com/1471-2105/14/194 (or
>> >> >> >> >> >> other
>> >> >> >> >> >> tools?
>> >> >> >> >> >> not
>> >> >> >> >> >> my
>> >> >> >> >> >> expertise). Do we have someone with capabilities in
>> >> >> >> >> >> this
>> >> >> >> >> >> area
>> >> >> >> >> >> on
>> >> >> >> >> >> this list?
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> > Thanks for these suggestions.
>> >> >> >> >> >
>> >> >> >> >> > I like the idea of linking into the scientific
>> >> >> >> >> > literature,
>> >> >> >> >> > initially as
>> >> >> >> >> > part of the 'Citation' section on each landing page
>> >> >> >> >> > rather
>> >> >> >> >> > than
>> >> >> >> >> > as
>> >> >> >> >> > a shield
>> >> >> >> >> > (maybe a shield in the long term). As Wolfgang mentions
>> >> >> >> >> > it
>> >> >> >> >> > is a
>> >> >> >> >> > little more
>> >> >> >> >> > challenging than a one-liner to match the information
>> >> >> >> >> > available
>> >> >> >> >> > from a
>> >> >> >> >> > CITATION file (or automatically generated) to an
>> >> >> >> >> > appropriate
>> >> >> >> >> > search
>> >> >> >> >> > in
>> >> >> >> >> > PubMed, and because citations are an important formal
>> >> >> >> >> > metric it
>> >> >> >> >> > seems
>> >> >> >> >> > important to get this more-or-less right.
>> >> >> >> >> >
>> >> >> >> >> > For what it's worth the more-or-less continuous stream
>> >> >> >> >> > of
>> >> >> >> >> > papers
>> >> >> >> >> > citing
>> >> >> >> >> > 'Biocondcutor' are listed at
>> >> >> >> >> >
>> >> >> >> >> >   http://bioconductor.org/help/publications/
>> >> >> >> >> >
>> >> >> >> >> > The links in the 'Literature Search' box query various
>> >> >> >> >> > resources
>> >> >> >> >> > for use
>> >> >> >> >> > of the term 'Bioconductor'.
>> >> >> >> >> >
>> >> >> >> >> > We have so far kept the distinction between 'available'
>> >> >> >> >> > and
>> >> >> >> >> > 'build',
>> >> >> >> >> > partly because builds sometimes fail for transient
>> >> >> >> >> > (e.g.,
>> >> >> >> >> > connectivity)
>> >> >> >> >> > reasons or, in devel, because of an incomplete check-in
>> >> >> >> >> > that
>> >> >> >> >> > does
>> >> >> >> >> > not
>> >> >> >> >> > compromise the end-user availability and functionality
>> >> >> >> >> > of
>> >> >> >> >> > the
>> >> >> >> >> > version
>> >> >> >> >> > available via biocLite().
>> >> >> >> >> >
>> >> >> >> >> > It's kind of amusing that (a) most of the information
>> >> >> >> >> > was
>> >> >> >> >> > already
>> >> >> >> >> > available, often on the landing page (like the links to
>> >> >> >> >> > build
>> >> >> >> >> > reports that
>> >> >> >> >> > Henrik mentions, or years in bioc), so the shields are
>> >> >> >> >> > serving
>> >> >> >> >> > just
>> >> >> >> >> > to
>> >> >> >> >> > emphasize these; and (b) the 'green' implies some-how
>> >> >> >> >> > 'good',
>> >> >> >> >> > but
>> >> >> >> >> > many of
>> >> >> >> >> > the shields (e.g., years in Bioc, posts, commits,
>> >> >> >> >> > downloads) are
>> >> >> >> >> > actually
>> >> >> >> >> > never not green. Maybe these shields should be white?
>> >> >> >> >> >
>> >> >> >> >> > Thanks again for the feedback; initial response seems
>> >> >> >> >> > to
>> >> >> >> >> > be
>> >> >> >> >> > positive.
>> >> >> >> >> >
>> >> >> >> >> > Martin
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >  PS  Martin you’ll like Fig. 2 of their paper.
>> >> >> >> >> >>
>> >> >> >> >> >> Wolfgang
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>  On May 9, 2015, at 19:15 GMT+2, Martin Morgan
>> >> >> >> >> >>  <mtmorgan at fredhutch.org>
>> >> >> >> >> >>> wrote:
>> >> >> >> >> >>>
>> >> >> >> >> >>> Bioc developers!
>> >> >> >> >> >>>
>> >> >> >> >> >>> It's important that our users be able to identify
>> >> >> >> >> >>> packages
>> >> >> >> >> >>> that
>> >> >> >> >> >>> are
>> >> >> >> >> >>> suitable for their research question. Obviously a
>> >> >> >> >> >>> first
>> >> >> >> >> >>> step
>> >> >> >> >> >>> is
>> >> >> >> >> >>> to identify
>> >> >> >> >> >>> packages in the appropriate research domain, for
>> >> >> >> >> >>> instance
>> >> >> >> >> >>> through
>> >> >> >> >> >>> biocViews.
>> >> >> >> >> >>>
>> >> >> >> >> >>>   http://bioconductor.org/packages/release/
>> >> >> >> >> >>>
>> >> >> >> >> >>> We'd like to help users further prioritize their
>> >> >> >> >> >>> efforts
>> >> >> >> >> >>> by
>> >> >> >> >> >>> summarizing
>> >> >> >> >> >>> use and usability. Metrics include:
>> >> >> >> >> >>>
>> >> >> >> >> >>> - Cross-platform availability -- biocLite()-able from
>> >> >> >> >> >>> all or
>> >> >> >> >> >>> only
>> >> >> >> >> >>> some
>> >> >> >> >> >>> platforms
>> >> >> >> >> >>> - Support forum activity -- questions and comments /
>> >> >> >> >> >>> responses, 6
>> >> >> >> >> >>> month
>> >> >> >> >> >>> window
>> >> >> >> >> >>> - Download percentile -- top 5, 20, 50%, or
>> >> >> >> >> >>> 'available'
>> >> >> >> >> >>> - Current build status -- errors or warnings on some
>> >> >> >> >> >>> or
>> >> >> >> >> >>> all
>> >> >> >> >> >>> platforms
>> >> >> >> >> >>> - Developer activity -- commits in the last 6 months
>> >> >> >> >> >>> - Historical presence -- years in Bioconductor
>> >> >> >> >> >>>
>> >> >> >> >> >>> Obviously the metrics are imperfect, so constructive
>> >> >> >> >> >>> feedback
>> >> >> >> >> >>> welcome --
>> >> >> >> >> >>> we think the above capture in a more-or-less
>> >> >> >> >> >>> objective
>> >> >> >> >> >>> and
>> >> >> >> >> >>> computable way
>> >> >> >> >> >>> the major axes influencing use and usability.
>> >> >> >> >> >>>
>> >> >> >> >> >>> We initially intend to prominently display 'shields'
>> >> >> >> >> >>> (small
>> >> >> >> >> >>> graphical
>> >> >> >> >> >>> icons) on package landing pages.
>> >> >> >> >> >>>
>> >> >> >> >> >>> Thanks in advance for your comments,
>> >> >> >> >> >>>
>> >> >> >> >> >>> Martin Morgan
>> >> >> >> >> >>> Bioconductor
>> >> >> >> >> >>> --
>> >> >> >> >> >>> Computational Biology / Fred Hutchinson Cancer
>> >> >> >> >> >>> Research
>> >> >> >> >> >>> Center
>> >> >> >> >> >>> 1100 Fairview Ave. N.
>> >> >> >> >> >>> PO Box 19024 Seattle, WA 98109
>> >> >> >> >> >>>
>> >> >> >> >> >>> Location: Arnold Building M1 B861
>> >> >> >> >> >>> Phone: (206) 667-2793
>> >> >> >> >> >>>
>> >> >> >> >> >>> _______________________________________________
>> >> >> >> >> >>> Bioc-devel at r-project.org mailing list
>> >> >> >> >> >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >> >> >> >> >>>
>> >> >> >> >> >>
>> >> >> >> >> >> _______________________________________________
>> >> >> >> >> >> Bioc-devel at r-project.org mailing list
>> >> >> >> >> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >
>> >> >> >> >> > --
>> >> >> >> >> > Computational Biology / Fred Hutchinson Cancer Research
>> >> >> >> >> > Center
>> >> >> >> >> > 1100 Fairview Ave. N.
>> >> >> >> >> > PO Box 19024 Seattle, WA 98109
>> >> >> >> >> >
>> >> >> >> >> > Location: Arnold Building M1 B861
>> >> >> >> >> > Phone: (206) 667-2793
>> >> >> >> >> >
>> >> >> >> >> > _______________________________________________
>> >> >> >> >> > Bioc-devel at r-project.org mailing list
>> >> >> >> >> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >> >> >> >> >
>> >> >> >> >>
>> >> >> >> >>       [[alternative HTML version deleted]]
>> >> >> >> >>
>> >> >> >> >> _______________________________________________
>> >> >> >> >> Bioc-devel at r-project.org mailing list
>> >> >> >> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >> >> >> >>
>> >> >> >> >
>> >> >> >> > _______________________________________________
>> >> >> >> > Bioc-devel at r-project.org mailing list
>> >> >> >> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >> >> >>
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > Bioc-devel at r-project.org mailing list
>> >> >> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >> >>
>> >> >> _______________________________________________
>> >> >> Bioc-devel at r-project.org mailing list
>> >> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >> >>
>> >> >
>> >> >         [[alternative HTML version deleted]]
>> >> >
>> >> > _______________________________________________
>> >> > Bioc-devel at r-project.org mailing list
>> >> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >>
>>