[Rd] Brainstorm: Alpha and Beta testing of R versions
hb at maths.lth.se
Sun Nov 6 01:43:04 CET 2005
Andrew Robinson wrote:
>On Fri, Nov 04, 2005 at 09:58:47AM +0100, Martin Maechler wrote:
>>[Mainly for R-foundation members; but kept in public for general
>I'll take up the invitation to brainstorm.
>As a user of R for a number of years, I'd really like to perform some
>useful service. I use a relatively obscure platform (FreeBSD) and I
>can compile code. I'd like to think that I'm in the target market for
>beta testing :). But, I'm timid. I do not feel, in general, that R
>core welcomes bug reports.
>I think that there are several things that could be tried to encourage
>more, and more useful, bug reports.
>1) Put the following text on the *front page* of the tracking system, so
> that it is seen before the reader clicks on "New Bug Report":
>"Before submitting a bug report, please read Chapter `R Bugs' of `The
>R FAQ'. It describes what a bug is and how to report a bug.
>If you are not sure whether you have observed a bug or not, it is a
>good idea to ask on the mailing list R-Help by sending an e-mail to
>r-help at stat.math.ethz.ch rather than submitting a bug report."
>(BTW is this true also for alpha/beta testing?)
>2) Try to use the structure of the reporting page to prompt good
> reporting. On the report page, summarize the key points of
> identifying and reporting a bug in a checklist format. Maybe even
> insist that the boxes be checked before allowing submission.
> Include seperate text boxes for description and sample code, to
> suggest that sample code is valued.
...and a optional field to select one or several packages related to the
bug. This is a good place to clarify that problems related to
third-party packages should not be reporter "here". Example HTML code:
Package(s) related to the bug, if applicable:<br>
(Bugs related to packages not listed below should <em>not</em> be
reported here. Instead, contact the package manager.)
<select name="packages" multiple>
<option value="">- - Select one or more packages related to the bug -
<option value="base">base (Base R functions)</option>
<option value="datasets">datasets (Base R datasets)</option>
<option value="grDevices">grDevices (Graphics devices for base and grid
<option value="graphics">graphics (R functions for base graphics)</option>
<option value="grid">grid (A rewrite of the graphics layout
capabilities, plus some support for interaction)</option>
<option value="methods">methods (Formally defined methods and classes
for R objects, plus other programming tools, as described in the Green
<option value="splines">splines (Regression spline functions and
<option value="stats">stats (R statistical functions)</option>
<option value="stats4">stats4 (Statistical functions using S4
<option value="tcltk">tcltk (Interface and language bindings to Tcl/Tk
<option value="tools">tools (Tools for package development and
<option value="utils">utils (R utility functions)</option>
>3) On either or both pages (and in FAQ), explain that thoughtful bug
> reports are valued and appreciated. Further, explain that bug
> reports that do not follow the protocol are less valuable, and take
> more time.
>4) Add checkboxes to the report page for alpha/beta. (I suggest this
> for the purposes of marketing, not organization.)
>5) On the report page, include hyperlinks to archived bug reports that
> were good. Do likewise with some artificial bug reports that are
>6) Add an intermediate, draft step for bug submission, to allow
> checking. If possible, include as part of this step an automated
> pattern matching call that identifies similarly texted bug reports,
> provides links to the reports, and invites a last-minute cross-check.
>7) Keep a list of people who report useful bugs in alpha/beta phase on
> the website. Many academics could point to it as evidence of
> community service.
>>In order to discourage an increased number of non-bug reports we
>>may have to also open a "hall of shame" though...
>8) I'm sure that you're being ironic! But I will take the point
> seriously, for what it's worth. I think that humiliating
> submitters who haven't followed the protocol is deleterious. It
> seems like almost every month we see someone get slapped on the
> wrist for not doing something the right way. Of course, it's
> frustrating that people aren't following the posting guide. But,
> why is that? Where is the breakdown? It might be interesting to
> try some follow-up (an exit interview!). If someone has failed to
> follow the protocol, perhaps we should try to find out why it was
> confusing, or if they just ignored it.
> The R-core is surrounded by, and serves, a community that comprises
> people who are not sufficiently good at what R-core does to be
> invited in to R-core. But, we're clearly interested in what R-core
> produces. Please don't assume that bug submissions that do not
> follow the R protocol are the consequence of deliberate
> To paraphrase Ian Fleming: Once is happenstance. Twice is
> incompetence. The third time, Mr. Bond, is enemy action. So, ...
>9) Publicly thank bug reporters whether their reports are useful or
> not. I just googled 'R-devel thank' and you figure prominently,
> Martin :).
More information about the R-devel