[Rd] Competing with one's own work

Fri Dec 3 20:00:26 CET 2010

Dear Doug,

Thank you for the response.

"constrOptim is in the stats package, not the base package."

Yes, I know, and I meant to say base *distribution* rather than base
package.  

"The burden of maintaining the code, fixing bugs or other infelicities, etc.
is on the package maintainer."

Of course.

"I don't see anything in what you are proposing that could not be
incorporated in a contributed package."

I agree, and it has already been done.  

What I am really asking is this: what is the rationale behind having a
package incorporated into the base distribution? 

Best,
Ravi.

-------------------------------------------------------
Ravi Varadhan, Ph.D.
Assistant Professor,
Division of Geriatric Medicine and Gerontology School of Medicine Johns
Hopkins University

Ph. (410) 502-2619
email: rvaradhan at jhmi.edu

-----Original Message-----
From: dmbates at gmail.com [mailto:dmbates at gmail.com] On Behalf Of Douglas
Bates
Sent: Friday, December 03, 2010 1:28 PM
To: Ravi Varadhan
Cc: Duncan Murdoch; nashjc at uottawa.ca; r-devel at r-project.org
Subject: Re: [Rd] Competing with one's own work

On Fri, Dec 3, 2010 at 11:01 AM, Ravi Varadhan <rvaradhan at jhmi.edu> wrote:
> Dear Duncan,

> What constitutes a convincing argument for making significant changes?
> Taking the example of optimization algorithms (say, for smooth objective
> functions), how does one make a convincing argument that a particular
class
> of algorithms are "better" than another class? This can be a difficult
task,
> but quite doable with good benchmarking practices.

> Supposing for the moment that such a convincing argument has been made, is
> that sufficient to get the R-core to act upon it?  Are there compelling
> factors other than just "algorithm A is better than algorithm B"?

> I'd think that the argument is relatively easy if the need for the change
is
> driven by consumer demand. But, even here I am not sure how to make an
> argument to the R-core to consider the big changes.  For example, there is
a
> reasonable demand for constrained (smooth) optimization algorithms in R
> (based on R-help queries).  Currently, there are only 3 packages that can
> handle this.  However, in the base distribution only `constrOptim'
function
> is provided, which cannot handle anything more than linear, inequality
> constraints.  I think that the base distribution needs to have a package
for
> constrained optimization that can handle linear/nonlinear and
> equality/inequality constraints.

constrOptim is in the stats package, not the base package.  Functions
that are already in the required packages are maintained by R core.
If you know of bugs in such functions you should report them.  Because
there is a heavy burden in maintaining the large corpus of software in
R and its required packages, additions are viewed skeptically,
Adopting new capabilities and new code in a required package like
stats means that some member of R core has to be willing to maintain
it.  If the capabilities can be incorporated in a contributed package
then that is the preferred method of extending R. The burden of
maintaining the code, fixing bugs or other infelicities, etc. is on
the package maintainer.

I don't see anything in what you are proposing that could not be
incorporated in a contributed package.

> John, thanks for raising an important issue.
>
> Thanks & Best,
> Ravi.
>
> -------------------------------------------------------
> Ravi Varadhan, Ph.D.
> Assistant Professor,
> Division of Geriatric Medicine and Gerontology School of Medicine Johns
> Hopkins University
>
> Ph. (410) 502-2619
> email: rvaradhan at jhmi.edu
>
>
> -----Original Message-----
> From: r-devel-bounces at r-project.org [mailto:r-devel-bounces at r-project.org]
> On Behalf Of Duncan Murdoch
> Sent: Friday, December 03, 2010 11:13 AM
> To: nashjc at uottawa.ca
> Cc: r-devel at r-project.org
> Subject: Re: [Rd] Competing with one's own work
>
> On 03/12/2010 10:57 AM, Prof. John C Nash wrote:
>> No, this is not about Rcpp, but a comment in that overly long discussion
> raised a question
>> that has been in my mind for a while.
>>
>> This is that one may have work that is used in R in the base
functionality
> and there are
>> improvements that should be incorporated.
>>
>> For me, this concerns the BFGS, Nelder-Mead and CG options of optim(),
> which are based on
>> the 1990 edition (Pascal codes) of my 1979 book "Compact numerical
> methods...", which were
>> themselves derived from other people's work. By the time Brian Ripley
took
> that work (with
>> permission, even though not strictly required. Thanks!) there were
already
> some
>> improvements to these same algorithms (mainly bounds and masks) in the
> BASIC codes of the
>> 1987 book by Mary Walker-Smith and I. However, BASIC to R is not
something
> I'd wish on
>> anyone.
>>
>> Now there are some R packages, including some I've been working on, that
> do offer
>> improvements on the optim() offerings. I would not say mine are yet fully
> ready for
>> incorporation into the base, but they are pretty close. Equally I think
> some of the tools
>> in the base should be deprecated and users encouraged to try other
> routines. It is also
>> getting more and more important that novice users be provided with
> sensible guidance and
>> robust default settings and choices. In many areas, users are faced with
> more choice than
>> is efficient for the majority of problems.
>>
>> My question is: How should such changes be suggested / assisted? It seems
> to me that this
>> is beyond a simple feature request. Some discussion on pros and cons
would
> be appropriate,
>> and those like myself who are familiar with particular tools can and
> should offer help.
>>
>> Alternatively, is there a document available in the style "Writing R
> Extensions" that has
>> a title like "How the R Base Packages are Updated"? A brief search was
> negative.
>>
>> I'm happy to compete with my own prior work to provide improvements. It
> would be nice to
>> see some of those improvements become the benchmark for further progress.
>
>
> There are answers at many different levels to your questions.  The
> simplest is that base packages are part of R, so they get updated when a
> member of R Core updates them, and the updates get released when a new
> version of R is released.
>
> So if you want a change, you need to convince a member of the core to
> make it.  Pointing out a bug is the easiest way to do this:  bugs
> usually get fixed quickly, if they are clearly demonstrated.
>
> If you want a bigger change, you need to make a convincing argument in
> favour of it.  If you pick a topic that is of particular interest to one
> core member, and you can convince him to make the change, then it will
> happen.  If pick some obscure topic that's not of interest to anyone,
> you'll need a very strong argument to make it interesting.  Part of any
> of these arguments is explaining why the change needs to be made to the
> base, why it can't just be published in a contributed package.  (That's
> why bug fixes are easy, and big additions to the base packages are not.)
>
> Duncan Murdoch
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>