[R] Loess with more than 4 predictors / offsets

[Louisell, Paul]

I apologize for clogging up inboxes, but I realized I needed to amend
what I said in my last comment below:

In fact, I'd like to specify that it be unconditionally linear, but with
estimated coefficients, _both an intercept and a slope_.

If the "offset" were only multiplied by a nonzero constant c, this would
have the effect of moving the whole response surface -log(c) units
parallel to the response axis in the scenario I outline below. This
would effectively give me the same thing I already have.


Paul Louisell
650-833-6254
ploua at allstate.com
Research Associate (Statistician)
Modeling & Data Analytics
ARPC


-----Original Message-----
From: Louisell, Paul 
Sent: Tuesday, January 23, 2007 12:40 PM
To: 'Prof Brian Ripley'
Cc: r-help at stat.math.ethz.ch
Subject: RE: [R] Loess with more than 4 predictors / offsets

In response to your questions:

I asked about including the offset for convenience; I currently put the
offset in by subtracting it from the response, just as you suggest. The
reason for including them is that I'm doing something slightly unusual
with loess:

I'm fitting loess to log((response+1)/offset) because the response is
actually a vector of counts. This is intended to give a rough
approximation to a Poisson regression; the reason for using loess is
that the mean response should be approximated by a Poisson process with
4 predictor variables which can be divided into 2 pairs, each pair of
which are geographic location coordinates. The two location pairs are
expected to exhibit strong interaction; hence, the reason for fitting
loess to all 4 predictors.

I'm aware of the curse of dimensionality, but I have a very large
dataset--over 600,000 observations. Since each pair of predictors
represents a point on a grid, I think Euclidean distance is probably a
good choice. And this brings me to the motivation for wanting to fit
with 5 predictors:

The offset is not _really_ an offset; it's just an approximation to what
the real offset should be. Hence, I'd rather include it as a predictor
than artificially force it to be included linearly with a coefficient of
1. I'm less concerned with linearity than I am with forcing the
coefficient. In fact, I'd like to specify that it be unconditionally
linear, but with an estimated coefficient. 

Thanks,


Paul Louisell
650-833-6254
ploua at allstate.com
Research Associate (Statistician)
Modeling & Data Analytics
ARPC

-----Original Message-----
From: Prof Brian Ripley [mailto:ripley at stats.ox.ac.uk] 
Sent: Monday, January 22, 2007 11:01 PM
To: Louisell, Paul
Cc: r-help at stat.math.ethz.ch
Subject: Re: [R] Loess with more than 4 predictors / offsets

On Mon, 22 Jan 2007, Louisell, Paul wrote:

> Hello,
>
> Does anyone know of an R version of loess that allows more than 4
> predictors and/or allows the specification of offsets? For that
matter,
> does anyone know of _any_ version of loess that does either of the
> things I mention?

Why would you want offsets in a regression?: just subtract them from the

lhs.  (R's lm has gained offsets by analogy with glm, but the S original

did not have them).  If you would be more comfortable working with them,

it would be very easy to create a modified version that supports them.

Also, have you heard of the 'curse of dimensionality'?  Localization
even 
to 4 dimensions is no longer really an appropriate term, and Euclidean 
distance will be the main determinant of 'local' and is quite arbitrary.


> Thanks,
>
> Paul Louisell
> 650-833-6254
> ploua at allstate.com
> Research Associate (Statistician)
> Modeling & Data Analytics
> ARPC
>
>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595