[Bioc-devel] Code quality and bug reports

Martin Morgan martin.morgan at roswellpark.org
Fri Jan 27 11:57:09 CET 2017 

On 01/27/2017 03:30 AM, Lluís Revilla wrote:
> Dear Valerie,
>
> When I talked about maintenance status I thought something on the line of
> this badges at http://www.repostatus.org/ :
> Maybe only the last three status are relevant to Bioconductor :
>
>   - Active: The project has reached a stable, usable state and is being
> actively developed.
>   - Inactive: The project has reached a stable, usable state but is no
> longer being actively developed; support/maintenance will be provided as
> time allows.
>   - Unsupported: The project has reached a stable, usable state but the
> author(s) have ceased all work on it. A new maintainer may be desired.

I'm supportive of a clearer set of labels to replace the current 'posts' 
shield.

We should not equate (new) development activity with level of support: 
there are other badges for that; one doesn't want development for 
development's sake; etc.

We have to be very careful that the tags (and current discussion) 
provide positive reinforcement to conscientious developers / community 
members, rather than serving to bully or otherwise intimidate developers 
(sometimes silence is a constructive response, and an effective use of 
time).

Iterating a little

   - Active (green): frequent support questions and answers
   - Mature (green): some support questions and answers
   - Inactive (orange):  some support questions unanswered
   - Unsupported (red): (more than a few?) support questions not answered
   - Unknown (blue): no support questions

Comments are ignored in the above scheme. Activity on packages with URL: 
or BugReports: fields in their DESCRIPTION file is ignored.  'frequent', 
'some', and 'few' could be defined based on available data.

Martin

>
> Tweaking a bit the description it could be used to classify the support
> given to a package. Given these categories and the information there is in
> the repositories and in support site I suggest the following:
>
> Giving the Active badge when there are commits (not done by the
> Bioconductor project) in 6 months. Excluding the Bioconductor team is done
> to prevent having packages as Active which are not, because the only change
> is the version number or minor changes done by the Bioconductor team (Also,
> is not the responsibility of the Bioconductor team to maintain the
> packages, is it?). Or when at least once every six months the package
> maintainer answers or comments a question (tagged with the package tag) in
> the support site if there is any.As an example edgeR would be Active, it
> has commits from the maintainers in the last 6 months.
>
> If a question tagged with a package tag is unanswered and the maintainer
> hasn't answered/commented in the last 6 months or there isn't any commit in
> the last 6 months the package (by the maintainer or other than the
> Bioconductor team) could be set to Inactive. CorMut would be Inactive and
> close to the Unsupported status.
>
> If there is any unanswered question tagged with a package tag and the
> maintainer hasn't answered/commented in the last year and there isn't any
> commit from the maintainer in the last 2 years, I would give the
> Unsupported tag to that package. topGO would be in that category.
>
> Once Unsupported status is reached the team could try to contact the
> maintainer to let him/her know that the maintainer position could be taken
> by somebody else willing to. Of course, if he/she makes commits or
> answers/comment questions in the support site to make the status of the
> package back to Active he/she could keep the maintainer status.
>
> This system could be along with the current End of Life Policy, or not, but
> gives an opportunity to the community of users to maintain a package they
> deem useful. It is a bit more complex but would guide much better on what
> packages are well supported and not only used. and those used but not
> supported could be saved from Deprecation.
>
> HTH,
>
> Lluís
>
> On 18 January 2017 at 17:52, Obenchain, Valerie <
> Valerie.Obenchain at roswellpark.org> wrote:
>
>> Hi,
>>
>> On 01/14/2017 03:01 AM, Lluís Revilla wrote:
>>> Dear Valerie and all,
>>>
>>> If I understood the page you kindly linked correctly, a package is
>> deprecated:
>>> 1) When it fails to build and check (unless it is fixed).
>>> 2) When the maintainer asks for it.
>>> 3) If the maintainer is unresponsive (I assume when a mail is not
>>> delivered) and(/or?) doesn't answers questions about the package (How
>>> is this tracked? Which is the threshold for unanswered questions, 1
>>> year? )
>> We contact maintainers when a package fails on the build system. There
>> isn't a set rule on the number of times contacted with no response
>> because there are so many exceptions, e.g., transferred maintainer ship,
>> away from email due to travel, etc. I'd say the average number of
>> contacts is 3 before getting the final 2 week notice.
>>
>>>
>>> In some packages, it seems the maintainers receive the mails, and the
>>> packages build and past the daily checks of Bioconductor, but there
>>> are bugs and issues with those packages that are left unanswered and
>>> unsolved in support.bioconductor.org. Those packages that are still
>>> functional and used but don't receive maintenance are then kept ? How
>>> should the community help to solve those issues?
>> A primary motivation for implementing badges on the landing pages was to
>> provide the "maintenance status" you mention below. The badge stats give
>> an idea of how active the maintainer is (posts, commits, coverage) as
>> well as the level of use by others (downloads). The 'posts' badge shows
>> support site activity over that past 6 months in terms of 4 fields:
>> tagged questions / average answers per question / average comments per
>> question / accepted answers.
>>
>> The CorMut package has no tagged questions:
>>
>>   http://www.bioconductor.org/packages/3.5/bioc/html/CorMut.html
>>
>> If Guangchuang had asked questions on the support site instead of
>> posting comments in a gist there would be a number of tagged questions
>> with no answers. This would give other users some data to go on - an
>> unresponsive maintainer of a package with no unit tests. In contrast, a
>> package like edgeR has an average of 1 answer and 3 comments for every
>> question asked:
>>
>>   http://www.bioconductor.org/packages/3.5/bioc/html/edgeR.html
>>
>> We don't have a system in place to remove packages from the repo based
>> on unanswered questions or bug reports. Ideally the combination of badge
>> stats and input from others on the support site (i.e., 'what package
>> would you recommend for ...') is sufficient to get direction on actively
>> maintained and well-thought-of packages.
>>
>> If there are suggestions for improving the badges please let us know.
>> There may also be parts of the package review process that could be
>> improved such as requiring unit tests instead of suggesting them. But
>> again, having unit tests does not guarantee that all (or the most
>> important) aspects of the package are tested.
>>
>> Valerie
>>
>>> Thanks,
>>>
>>> Lluís
>>>
>>> On 6 January 2017 at 18:56, Obenchain, Valerie
>>> <Valerie.Obenchain at roswellpark.org> wrote:
>>>> On 01/04/2017 07:53 AM, Lluís Revilla wrote:
>>>>> Dear Guangchuang and all,
>>>>>
>>>>> Quality of the packages has been a preoccupation of the project from
>>>>> the  beginning  (see
>>>>> https://stat.ethz.ch/pipermail/bioc-devel/2014-May/005810.html for
>>>>> more references and other discussions about bug reports.) Despite not
>>>>> being in a goal of the project, it is necessary to do a reproducible
>>>>> research, which it is a goal: "To further scientific understanding by
>>>>> producing high-quality documentation and reproducible research.".
>>>>> Although it seems that this discussion appears periodically
>>>>> Bioconductor I don't know any solution in the project.
>>>>>
>>>>> I have never submitted a package to Bioconductor or CRAN, and I am
>>>>> quite new to R (and is my first mail to the list), but one thing that
>>>>> I keep thinking (before publishing a package) is the maintainability
>>>>> of it. I don't know how much time/desires will I have to dedicate to a
>>>>> package (if it is accepted) in the future, but at the same time I want
>>>>> to make useful code to be used in further research beyond my own
>>>>> project.
>>>>>
>>>>> However the Bioconductor core team may be already too busy to deal
>>>>> with the issues of all packages. Maybe it would be better to bring
>>>>> CRAN's policy to orphan packages (see:
>>>>> https://cran.r-project.org/src/contrib/Orphaned/README):
>>>> Bioconductor does have a system for dealing with orphaned packages
>>>> called End of Life:
>>>>
>>>>   http://www.bioconductor.org/developers/package-end-of-life/
>>>>
>>>> Deprecated packages have a strikethrough the name on the build reports,
>>>> e.g., the saps package,
>>>>
>>>>   http://www.bioconductor.org/checkResults/devel/bioc-LATEST/
>>>>
>>>>
>>>> Valerie
>>>>
>>>>> "Possible reasons for orphanizing a package:
>>>>>
>>>>> 1) The current maintainer actively wants to orphanize the package,
>>>>>    e.g., due to no longer having time or interest to act as package
>>>>>    maintainer.
>>>>>
>>>>> 2) Emails sent to the current maintainer by the CRAN admins bounced, or
>>>>>    were not answered for longer periods of time.
>>>>> "
>>>>> Example of an orhpan package is clusterRepro.
>>>>>
>>>>> To orphanize a package the current maintainer could post it here on
>>>>> the devel list and ask for a maintainer on the support site, and it is
>>>>> his/her decision to whom is passed the responsibility.
>>>>> Otherwise, maybe the limit of time without answering mails/posts in
>>>>> support could be 3 months/6 months. (I don't know the CRAN decision
>>>>> when not answering mails)
>>>>>
>>>>> Once the orphaned status is reached maybe however who wants could send
>>>>> patches or take the maintenance of the package for another 3 months or
>>>>> more.
>>>>>
>>>>> This status would not make bugs easier to fix or control, but could
>>>>> mark that a package is not in the best maintenance status.
>>>>>
>>>>> Hope this helps,
>>>>>
>>>>> Lluís
>>>>>
>>>>>
>>>>> ----
>>>>> Date: Wed, 4 Jan 2017 15:38:53 +0800
>>>>> From: "Yu, Guangchuang" <gcyu at connect.hku.hk>
>>>>> To: bioc-devel <bioc-devel at r-project.org>
>>>>> Subject: [Bioc-devel] report package issue to Bioconductor
>>>>> Message-ID:
>>>>>         <CAEF2yOdEQ9LOA3nFbLBvNddtLTKMZ-YiVkCf7UMJRRpfQP7Lmw at mail.
>> gmail.com>
>>>>> Content-Type: text/plain; charset="UTF-8"
>>>>>
>>>>> Dear all,
>>>>>
>>>>> Some packages never updated after they publish a paper, and they just
>>>>> ignore bug report.
>>>>>
>>>>> I think we need somewhere, maybe on github, to post code review and
>>>>> Bioconductor core team can take action if maintainer fail to fix issue.
>>>>>
>>>>> Here is a quick look of the CorMut package:
>>>>> https://gist.github.com/GuangchuangYu/91b3396c7e49ab42c565a9cda3c35e18
>> .
>>>>>
>>>>> There should be more issues than I can found with quick look of the
>> source
>>>>> code.
>>>>>
>>>>> Best wishes,
>>>>> Guangchuang
>>>>> ?
>>>>> --
>>>>> --~--~---------~--~----~------------~-------~--~----~
>>>>> Guangchuang Yu, PhD Candidate
>>>>> State Key Laboratory of Emerging Infectious Diseases
>>>>> School of Public Health
>>>>> The University of Hong Kong
>>>>> Hong Kong SAR, China
>>>>> www: https://guangchuangyu.github.io
>>>>> -~----------~----~----~----~------~----~------~--~---
>>>>>
>>>>> _______________________________________________
>>>>> Bioc-devel at r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>>>
>>>>
>>>>
>>>> This email message may contain legally privileged and/or confidential
>> information.  If you are not the intended recipient(s), or the employee or
>> agent responsible for the delivery of this message to the intended
>> recipient(s), you are hereby notified that any disclosure, copying,
>> distribution, or use of this email message is prohibited.  If you have
>> received this message in error, please notify the sender immediately by
>> e-mail and delete this email message from your computer. Thank you.
>>
>>
>>
>>
>> This email message may contain legally privileged and/or confidential
>> information.  If you are not the intended recipient(s), or the employee or
>> agent responsible for the delivery of this message to the intended
>> recipient(s), you are hereby notified that any disclosure, copying,
>> distribution, or use of this email message is prohibited.  If you have
>> received this message in error, please notify the sender immediately by
>> e-mail and delete this email message from your computer. Thank you.
>>
>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>


This email message may contain legally privileged and/or...{{dropped:2}}

More information about the Bioc-devel mailing list