[Rd] Automatic Differentiation for R

[Forth, Shaun]

Thanks for your interest in this topic.  

In the AD community we are used to dealing with essentially arbitrarily complex computer code.  Some Fortran and C activities have looked at applications such as global climate models and computational fluid dynamics codes for aerospace design.  Fortran and C have relatively small intrinsic operations and function set.  I've done quite of lot or work with Matlab and had to consider "intrinsics" such as linear solves and determinants; this is probably more like the R approach. Matlab software is used by industry and commerce for involved computations with loops, branches and some collaborators have considered OO programs.

I look forward to hearing about R at the Oxford AD meeting and perhaps seeing if we can start to explore some efficient AD implementations.

Shaun





##################################################################### 
Dr Shaun Forth 
Applied Mathematics & Scientific Computation
Cranfield Defence and Security 
Cranfield University, Shrivenham Campus
Swindon SN6 8LA, England 
tel: +44 (0)1793 785311 
fax: +44 (0)1793 784196 
email: S.A.Forth at cranfield.ac.uk 
http://www.amorg.co.uk 
##################################################################### 

>-----Original Message-----
>From: Gabor Grothendieck [mailto:ggrothendieck at gmail.com] 
>Sent: Tuesday, May 19, 2009 2:58 PM
>To: Martin Maechler
>Cc: John C Nash; r-devel at r-project.org; Forth, Shaun
>Subject: Re: [Rd] Automatic Differentiation for R
>
>On Tue, May 19, 2009 at 9:08 AM, Martin Maechler
><maechler at stat.math.ethz.ch> wrote:
>> [MM stumbling over on old thread ... he'd be interested]
>>
>>>>>>> "GaGr" == Gabor Grothendieck <ggrothendieck at gmail.com>
>>>>>>>     on Wed, 15 Apr 2009 09:53:18 -0400 writes:
>>
>>    GaGr> Not sure if this is sufficient for your needs but R 
>does include symbolic
>>    GaGr> differentiation, see ?D, and the Ryacas and rSymPy
>>    GaGr> packages interface R to the yacas and sympy computer algebra
>>    GaGr> systems (CAS) and those system include symbolic 
>differentiation.
>>
>> No, symbolic differentiation is not enough.
>> Automatic Differentiation (AD) is something much more general (in one
>> way) and much less mathematical from  a classical view point:
>> But then, AD is much more generally useful for minimization 
>as, basically,
>> the input is an R function
>>    f(x)               {with x multidimensional}
>> or  f(x1,x2, ..., xp)  {with scalar x1, x2, ..}
>> and the output is again an R function
>> which computes f() and all {or just selected} partial
>> derivatives  d f / d{xi}.
>>
>> Now consider that the function f()  can contain if() and while()
>> clauses and conceptually ever language feature of R.
>> In practice, I'm pretty sure the list of features would have to
>> be restricted, similarly as they'd have to for an R compiler to
>> be feasible.
>>
>> I agree that  AD for R would be very nice and could be very
>> useful.
>> I'd also be interested to help AD people learn the S4 classes
>> and methods (hoping that it's close enough to what they call
>> "operator overloading" something I'd presume to be less general
>> than the powerful S4 class/methods system).
>
>The overloading facilities present have already been discussed in
>this thread including a complete illustration of using them for the
>problem at hand.
>
>rSymPy and Ryacas both support overloading.
>
>Ryacas also supports automatic differentiation
>of one line R functions but its not fully developed and very limited.
>See demo("Ryacas-Function") which shows differentiation of the
>Burr CDF to get the PDF.
>
>Here are a few more simpler examples to illustrate overloading in
>these packages.
>
>> library(Ryacas)
>Loading required package: XML
>> x <- Sym("x")
>> x+x
>[1] "Starting Yacas!"
>expression(2 * x)
>
>> library(rSymPy)
>Loading required package: rJava
>> source("http://rsympy.googlecode.com/svn/trunk/R/Sym.R")
>> y <- Sym(sympy("var('y')"))
>> y+y
>[1] "2*y"
>
>Check the home pages of the packages for more info.
>