[R] Non-Linear Regression (Cobb-Douglas and C.E.S)

[Spencer Graves]

      Given enough data, the choice between the two models can be made 
in part by plotting the residuals vs. the predicted:  or vs. 
log(predicted):  Suppose the "true" model was

      log(Y) = log(ALPHA) +(BETA1)*log(L) + (BETA2)*log(K) + err,

where err is independent, normal with constant variance s.e^2.  This is 
is equivalent to

      Y = ALPHA * (L^(BETA1)) * (K^(BETA2))*exp(err)

      Suppose you fit with nls

      Y = ALPHA * (L^(BETA1)) * (K^(BETA2)) + Err,

assuming Err is independent, normal with constant variance s.E^2.  If Y 
ranges over less than a factor of 2 or 5, it probably doesn't matter 
much which model you use.  However, if Y ranges over 2 or more orders of 
magnitude, then a plot of residuals or abs(residuals) vs. predicted (or 
vs. log(predicted)) should show how the residuals get larger as the 
predictions increase. 

      For example, suppose we know that BETA1 = BETA2 = 1, K = 1, and L 
ranges from 0.001 to 1000, and we fit

      Y = ALPHA*K + Err,

when the "truth" is

      log(Y) = log(ALPHA) + log(L) + err,

with err ~ N(0, 1).  Then the following simulation shows what we want: 

log10.L <- seq(-3, 3, .2)
DF <- data.frame(L=10^log10.L, 
Y=exp(log(10)*log10.L+rnorm(length(log10.L))))

fitLin <- lm(Y ~ L-1, DF)
fitLog <- lm(log(Y/L) ~ 1, DF)

plot(DF$L, abs(resid(fitLin)), log="xy")
plot(DF$L, abs(resid(fitLog)), log="xy")
qqnorm(resid(fitLin), datax=TRUE)
qqnorm(resid(fitLog), datax=TRUE)

      The difference between resid(fitLin) and resid(fitLog) is dramatic. 

      Now suppose we modify the simulation by replacing the first 
lot10.L and DF as follows: 

log10.L <- seq(100, 100.1, length=31)
DF <- data.frame(L=10^log10.L, 
Y=exp(log(10)*log10.L+0.01*rnorm(length(log10.L))))

      Then the difference between resid(fitLin) and resid(fitLog) is 
hardly noticeable. 

      Beyond this, extensive experience with economic data indicates 
that many economic quantities are lognormally distrributed, provided the 
logic of the situation says the numbers should always be positive.  To 
understand this, consider for example the money spent each week by a 
small child with an allowance of $3 per week.  Suppose she spends 
roughly $3 per week, a little less some weeks and a little more in 
others.  Her expendatures may vary over a 3 month period from $2.37 to 
$3.69, just over 10%, 63 cents low one week, 69 cents high another.  Now 
compare this with the expendatures of a $3,000,000 per year business.  
If their business is stable over a decade, their expendatures may vary 
from $2,370,000 in their slowest year in that decade to $3,690,000 at 
the max, just over 10%.  However, that 10% is NOT 63 cents low in one 
case but $630,000. 
    
      Hope this helps.  spencer graves

Sundar Dorai-Raj wrote:

>
>
> Mohammad Ehsanul Karim wrote:
>
>> Dear Sundar Dorai-Raj,
>>
>> Thank you very much for mentioning to exponentiate ALPHA.
>>
>> However, so far i understand that the parameters in the non-linear 
>> equation
>> Y = ALPHA * (L^(BETA1)) * (K^(BETA2))
>> and the coefficients of log(L) and log(K) of the following equation 
>> (after linearizing)
>> log(Y) = log(ALPHA) +(BETA1)*log(L) + (BETA2)*log(K)
>> should be the same when estimated from either equation. Is it true? 
>> If it is, then why the estimates of the two procedure (see below) are 
>> different? Can you please explain it?
>> -----------------------------
>>  > coef(lm(log(Y)~log(L)+log(K), data=klein.data))
>>
>> (Intercept)      log(L)      log(K)
>>  -3.6529493   1.0376775   0.7187662
>> -----------------------------
>>  > nls(Y~ALPHA * (L^(BETA1)) * (K^(BETA2)), data=klein.data, start = 
>> c(ALPHA=exp(-3.6529493),BETA1=1.0376775,BETA2 = 0.7187662), trace = 
>> TRUE)
>>
>> Nonlinear regression model
>>   model:  Y ~ ALPHA * (L^(BETA1)) * (K^(BETA2))
>>    data:  klein.data
>>       ALPHA       BETA1       BETA2
>> 0.003120991 0.414100040 1.513546235
>>  residual sum-of-squares:  3128.245
>> -----------------------------
>>
>
> Not necessarily. In the first model, you're minimizing:
>
> sum((log(Y) - log(Yhat))^2)
>
> because the nonlinear model you're fitting is:
>
> Y = ALPHA * L^BETA1 * K^BETA2 * ERROR
> log(Y) = log(ALPHA) + BETA1 * log(L) + BETA2 * log(K) + log(ERROR)
>
> Note the multiplicative error structure. In the second model you're 
> mininmizing
>
> sum((Y - Yhat)^2)
>
> because the nonlinear model you're fitting is
>
> Y = ALPHA * L^BETA1 * K^BETA2 + ERROR
>
> Note the additive error structure. Different error structures, 
> different parameter estimates.
>
> Also, the residual sums of squares for the nls fit is smaller, 
> although I'm not sure whether this comparison is really fair:
>
> klein.lm <- lm(log(Y) ~ log(L) + log(K))
> # `start' is not shown here but can be copied from above
> klein.nls <- nls(Y ~ ALPHA * L^BETA1 * K^BETA2, data = klein.data,
>                  start = start, trace = TRUE)
> rss.lm <- sum((Y - exp(fitted(klein.lm)))^2) # 3861.147
> rss.nls <- sum((Y - fitted(klein.nls))^2) # 3128.245
>
> Now, which one do you use? Depends on whether you believe you have 
> multiplicative errors (use lm) or additive errors (use nls).
>
> --sundar
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! 
> http://www.R-project.org/posting-guide.html