[Rd] Automatic Differentiation for R

[John C Nash]

In efforts to improve optimization tools for R, one of my
interests has been getting automatic differentiation capabilities
so that analytic rather than numerical derivatives can be used. They
would be helpful in several other areas besides optimization, My timings 
show
factors of the order of 1000s in time improvements by avoiding
numerical derivatives in some cases.

There has been some work in this e.g., 
http://code.google.com/p/pbs-software/
is an R interface to ADMB (Automatic Differentiation Model Builder). 
However,
as far as I can see, this is directed essentially to nonlinear least 
squares modelling,
an important but not general problem.

Tom Coleman of Waterloo responded favourably with some advice, but the most
enthusiastic answer came from Shaun Forth, which I have included below. 
I read
this as an opportunity to develop what could be a profitable 
collaboration with
the AD community. Unfortunately, I cannot take up the invitation to join 
the AD
folk in Oxford due to a pre-existing obligation. Nor am I more than a 
complete
novice with S3 and S4 classes etc. I am, nevertheless, willing to help 
organize
the effort e.g., do some of the communications, chasing grant money, getting
Google Summer of Code applications filled in etc.

Can the R community come up with a few people who can provide the AD
workers with appropriate information? If so, is there a reasonable chance to
generate sufficient funding for a student? I suspect the answer in both 
cases
is yes, but that we need some form of "booster cables" to get things going.
(In Canada, booster cables are used to get cars started in winter by 
connecting
a running vehicle's battery to that of a dead one.)

I suggest communications off-list until there is progress to report. 
Possibly
there is a better forum for this -- suggestions welcome.

John Nash

---- included msg from Shaun Forth ---

Hi John,
My computational statistics colleague Trevor Ringrose has asked me to
consider AD in R in the past. As you may or may not be aware AD is
implemented in one of two ways: overloading using OO features of the
target language, or source transformation using compiler tools (after
several man years of development) to read in the target code and spit
out the differentiated code. Last time I looked I didn't think the
object oriented features of R were up to overloading but on checking
today I can see that this might now be possible (I can see overloading
of arithmetic operators and functions for example now which I didn't see
last time).  

I'd certainly be interested in following this up particularly on the
overloading side but would need to get funding for a PhD student to do
the graft. It would be particularly interesting doing this in an open
source language because we could then perhaps tweak some of the core
language features if they didn't seamlessly support the AD (we can't do
this in Matlab and that is a pain!).

My immediate suggestion is that you, or some other more local (to UK) R
expert talks at the next European AD workshop in Oxford 
http://www.autodiff.org/?module=Workshops&submenu=EuroAD/8/main
We're a very friendly group and I'm sure there are others who might like
to tackle R or perhaps we could put together a multigroup project. If
someone could give a talk on R, its language features including the OO
aspects, and some optimisation examples with associated code, the group
there would be able to give you the best feedback on the planet on the
possibilities.

Please do treat this as a positive response and let's keep in touch on
this.

Regards

Shaun


####################################################################
Dr Shaun Forth
Applied Mathematics & Scientific Computation
Cranfield Defence and Security 
Cranfield University, Shrivenham Campus 
Swindon SN6 8LA, England
tel: +44 (0)1793 785311
fax: +44 (0)1793 784196
email: S.A.Forth at cranfield.ac.uk
http://www.amorg.co.uk
#####################################################################