[Bioc-devel] Use and Usability metrics / shields

Leonardo Collado Torres lcollado at jhu.edu
Tue May 19 21:37:18 CEST 2015 

Regarding the 'posts' tag, I can see that it includes a "closed
questions" component. For example,
http://www.bioconductor.org/packages/release/bioc/html/derfinder.html
is 3/1/9/0 right now meaning that 0 questions are closed. From
https://support.bioconductor.org/info/faq/, only moderators can close
questions. That seems like quite a bit of work for the moderators. So
maybe it would be best to drop the "closed questions" component. Or
alternatively, can the author of a package moderate the posts that
have a tag corresponding to their package?

As for 'build: warnings', it seems like it will show for some devel
packages all the time. For example,
http://www.bioconductor.org/packages/devel/bioc/html/regionReport.html
always has a warning in the Windows build machine due to a mismatch in
the version of Rtools installed.

I do like these changes and the addition of shields =)


On Thu, May 14, 2015 at 10:56 AM, Dan Tenenbaum <dtenenba at fredhutch.org> wrote:
>
> ----- Original Message -----
>> From: "Jim Hester" <james.f.hester at gmail.com>
>> To: "Martin Morgan" <mtmorgan at fredhutch.org>
>> Cc: bioc-devel at r-project.org
>> Sent: Thursday, May 14, 2015 7:53:03 AM
>> Subject: Re: [Bioc-devel] Use and Usability metrics / shields
>>
>> The common shield convention is to use blue or orange when the
>> information
>> is not qualitatively good or bad, but the color choice is just
>> subjective
>> in the end.
>
> It does seem though that we should indicate the non-changing nature of these shields with some kind of color change. Perhaps we can come up with one that works with the other design elements on the page.
>
> BTW, the 'posts' tag does change color; if there are 0 posts tagged with a package name, the shield is yellow; otherwise it's green.
>
> Dan
>
>
>>
>> On Thu, May 14, 2015 at 10:47 AM, Martin Morgan
>> <mtmorgan at fredhutch.org>
>> wrote:
>>
>> > On 05/10/2015 11:39 AM, COMMO Frederic wrote:
>> >
>> >> Dear Martin,
>> >>
>> >> All of these suggestions sound good.
>> >>
>> >> Wolfgang's suggestion regarding possible associated papers might
>> >> be also
>> >> great.
>> >>
>> >> Another useful information would be to point to other publications
>> >> where
>> >> a given package was used, and cited.
>> >> I don't know if it's technically possible, but it would be greatly
>> >> informative to know how frequently a package is used, and how it
>> >> performs,
>> >> in real contexts.
>> >>
>> >> Frederic Commo
>> >> Bioinformatics, U981
>> >> Gustave Roussy
>> >>
>> >> ________________________________________
>> >> De : Bioc-devel [bioc-devel-bounces at r-project.org] de la part de
>> >> Wolfgang Huber [whuber at embl.de]
>> >> Date d'envoi : samedi 9 mai 2015 19:57
>> >> À : Martin Morgan
>> >> Cc: bioc-devel at r-project.org
>> >> Objet : Re: [Bioc-devel] Use and Usability metrics / shields
>> >>
>> >> Dear Martin
>> >>
>> >> great idea.
>> >> "Current build status” could perhaps be wrapped with
>> >> "Cross-platform
>> >> availability” into some sort of “Availability / Accessibility”?
>> >>
>> >> I wonder how informative it would be to make metrics such as
>> >> (i) citations of the associated paper
>> >> (ii) full-text mentions e.g. in PubmedCentral
>> >> actually useful. (i) could be flawed if package and paper are
>> >> diverged;
>> >> (ii) would require good disambiguation, e.g. like bioNerDS
>> >> http://www.biomedcentral.com/1471-2105/14/194 (or other tools? not
>> >> my
>> >> expertise). Do we have someone with capabilities in this area on
>> >> this list?
>> >>
>> >>
>> > Thanks for these suggestions.
>> >
>> > I like the idea of linking into the scientific literature,
>> > initially as
>> > part of the 'Citation' section on each landing page rather than as
>> > a shield
>> > (maybe a shield in the long term). As Wolfgang mentions it is a
>> > little more
>> > challenging than a one-liner to match the information available
>> > from a
>> > CITATION file (or automatically generated) to an appropriate search
>> > in
>> > PubMed, and because citations are an important formal metric it
>> > seems
>> > important to get this more-or-less right.
>> >
>> > For what it's worth the more-or-less continuous stream of papers
>> > citing
>> > 'Biocondcutor' are listed at
>> >
>> >   http://bioconductor.org/help/publications/
>> >
>> > The links in the 'Literature Search' box query various resources
>> > for use
>> > of the term 'Bioconductor'.
>> >
>> > We have so far kept the distinction between 'available' and
>> > 'build',
>> > partly because builds sometimes fail for transient (e.g.,
>> > connectivity)
>> > reasons or, in devel, because of an incomplete check-in that does
>> > not
>> > compromise the end-user availability and functionality of the
>> > version
>> > available via biocLite().
>> >
>> > It's kind of amusing that (a) most of the information was already
>> > available, often on the landing page (like the links to build
>> > reports that
>> > Henrik mentions, or years in bioc), so the shields are serving just
>> > to
>> > emphasize these; and (b) the 'green' implies some-how 'good', but
>> > many of
>> > the shields (e.g., years in Bioc, posts, commits, downloads) are
>> > actually
>> > never not green. Maybe these shields should be white?
>> >
>> > Thanks again for the feedback; initial response seems to be
>> > positive.
>> >
>> > Martin
>> >
>> >
>> >  PS  Martin you’ll like Fig. 2 of their paper.
>> >>
>> >> Wolfgang
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>  On May 9, 2015, at 19:15 GMT+2, Martin Morgan
>> >>  <mtmorgan at fredhutch.org>
>> >>> wrote:
>> >>>
>> >>> Bioc developers!
>> >>>
>> >>> It's important that our users be able to identify packages that
>> >>> are
>> >>> suitable for their research question. Obviously a first step is
>> >>> to identify
>> >>> packages in the appropriate research domain, for instance through
>> >>> biocViews.
>> >>>
>> >>>   http://bioconductor.org/packages/release/
>> >>>
>> >>> We'd like to help users further prioritize their efforts by
>> >>> summarizing
>> >>> use and usability. Metrics include:
>> >>>
>> >>> - Cross-platform availability -- biocLite()-able from all or only
>> >>> some
>> >>> platforms
>> >>> - Support forum activity -- questions and comments / responses, 6
>> >>> month
>> >>> window
>> >>> - Download percentile -- top 5, 20, 50%, or 'available'
>> >>> - Current build status -- errors or warnings on some or all
>> >>> platforms
>> >>> - Developer activity -- commits in the last 6 months
>> >>> - Historical presence -- years in Bioconductor
>> >>>
>> >>> Obviously the metrics are imperfect, so constructive feedback
>> >>> welcome --
>> >>> we think the above capture in a more-or-less objective and
>> >>> computable way
>> >>> the major axes influencing use and usability.
>> >>>
>> >>> We initially intend to prominently display 'shields' (small
>> >>> graphical
>> >>> icons) on package landing pages.
>> >>>
>> >>> Thanks in advance for your comments,
>> >>>
>> >>> Martin Morgan
>> >>> Bioconductor
>> >>> --
>> >>> Computational Biology / Fred Hutchinson Cancer Research Center
>> >>> 1100 Fairview Ave. N.
>> >>> PO Box 19024 Seattle, WA 98109
>> >>>
>> >>> Location: Arnold Building M1 B861
>> >>> Phone: (206) 667-2793
>> >>>
>> >>> _______________________________________________
>> >>> Bioc-devel at r-project.org mailing list
>> >>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >>>
>> >>
>> >> _______________________________________________
>> >> Bioc-devel at r-project.org mailing list
>> >> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >>
>> >>
>> >
>> > --
>> > Computational Biology / Fred Hutchinson Cancer Research Center
>> > 1100 Fairview Ave. N.
>> > PO Box 19024 Seattle, WA 98109
>> >
>> > Location: Arnold Building M1 B861
>> > Phone: (206) 667-2793
>> >
>> > _______________________________________________
>> > Bioc-devel at r-project.org mailing list
>> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >
>>
>>       [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioc-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel

More information about the Bioc-devel mailing list