[R] need help with smooth.spline

[roger koenker]

Another option for fitting smoothing spline models to data with abrupt  
changes
in derivative is the total variation penalty methods that are  
incorporated into
the R package "nprq" .   The models there are fitting piecewise linear  
models
both univariate and bivariate components are allowed, but the roughness
penalty is total variation of the derivative, or gradient in the  
bivariate case,
so sharp kinks are permitted.


url:	www.econ.uiuc.edu/~roger        	Roger Koenker
email	rkoenker at uiuc.edu			Department of Economics
vox: 	217-333-4558				University of Illinois
fax:   	217-244-6678				Champaign, IL 61820

On Mar 4, 2004, at 8:30 AM, Liaw, Andy wrote:

> Hi Carlisle,
>
> If I understand you correctly, the problem is smooth.spline() not  
> handling
> sharp jump(s), right?  If so, it's probably easier to try something  
> that can
> handle such features.  Wavelet `denoising' (as opposed to `smoothing',  
> and
> available in the wavethresh package) is well known for being able to  
> handle
> abrupt changes (very `spatially adaptive').  Other things you might  
> consider
> are mars() in the `mda' package (which fits splines in an adaptive  
> fashion)
> and locfit() in the `locfit' package.  For locfit, you will want to  
> specify
> local smoothing parameter selection, via a call like
>
>   locfit(..., alpha=c(0, 0, 2), acri="cp")
>
> You might need to play with the `2' a bit to get the right amount of
> smoothing.  The details are in Loader's book `Local regression and
> Likelihood'.
>
> HTH,
> Andy
>
>> From: W. C. Thacker
>>
>> Andy,
>>
>> As the data are often noisy, smoothing splines should be appropriate.
>>
>> The first example profile shows an isothermal (constant temperature)
>> layer in the upper ocean followed by a sharp thermocline (large
>> temperature gradient), but there are relatively few observations
>> defining this sharp transition.  In this case simple linear
>> interpolation works fine, but smooth.spline() with all defaults gives
>> an absolutely absurd value in the isothermal layer.  With all.knots =
>> TRUE, the values in the isothermal layer are much better but still
>> peculiar.
>>
>> Given the sampling and the data, is it possible to get smooth.spline()
>> do better?  If so, would that adversely impact its performance for
>> other cases?  (There are thousands of profiles.)  If not, is there a
>> simp[le way to select cases that smooth.spline() should not be
>> expected to handle, so they can be treated separately?
>>
>> Thanks,
>>
>> Carlisle
>>
>> "Liaw, Andy" wrote:
>>>
>>> If you really want interpolation, should you be using
>> spline() rather than
>>> smooth.spline()?  The later is for smoothing data observed
>> with noise, not
>>> for interpolation.
>>>
>>> Andy
>>>
>>>> From: W. C. Thacker
>>>>
>>>> Dear R listers,
>>>>
>>>> When using smooth.spline to interpolate data, results are
>> generally
>>>> good.  However, some cases produce totally unreasonable results.
>>>>
>>>> The data are values of pressure, temperature, and salinity from a
>>>> probe that is lowered into the ocean, and the objective is to
>>>> interpolate temperature and salinity to specified
>> pressures.  While
>>>> smooth.spline provides excellent values at the observed pressures,
>>>> there are cases when the values at the desired pressures are
>>>> unusable.  A dataframe with four such profiles, indicated
>> by values of
>>>> id, is attached.  My target values for pressure are
>> seq(25,1600,25),
>>>> but 1:500 is also interesting.
>>>>
>>>> Setting all.knots = TRUE helps, but it would be nice to
>> be able to do
>>>> better.
>>>>
>>>> Any suggestions?
>>>>
>>>> Thanks,
>>>>
>>>> Carlisle
>>>>
>>>>> version
>>>>          _
>>>> platform sparc-sun-solaris2.9
>>>> arch     sparc
>>>> os       solaris2.9
>>>> system   sparc, solaris2.9
>>>> status
>>>> major    1
>>>> minor    8.0
>>>> year     2003
>>>> month    10
>>>> day      08
>>>> language R
>>>>
>>>>
>>>> --
>>>>
>>>> William Carlisle Thacker
>>>>
>>>> Atlantic Oceanographic and Meteorological Laboratory
>>>> 4301 Rickenbacker Causeway, Miami, Florida 33149 USA
>>>> Office: (305) 361-4323           Fax: (305) 361-4392
>>>>
>>>> "Too many have dispensed with generosity
>>>>      in order to practice charity."     Albert Camus
>>>>
>>>> ______________________________________________
>>>> R-help at stat.math.ethz.ch mailing list
>>>> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide!
>>>> http://www.R-project.org/posting-guide.html
>>>>
>>>>
>>>
>>>
>> --------------------------------------------------------------
>> ----------------
>>> Notice:  This e-mail message, together with any
>> attachments,...{{dropped}}
>>>
>>> ______________________________________________
>>> R-help at stat.math.ethz.ch mailing list
>>> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide!
>> http://www.R-project.org/posting-guide.html
>>
>> --  
>>
>> William
>> Carlisle Thacker
>>
>> Atlantic Oceanographic and Meteorological Laboratory
>> 4301 Rickenbacker Causeway, Miami, Florida 33149 USA
>> Office: (305) 361-4323           Fax: (305) 361-4392
>>
>> "Too many have dispensed with generosity
>>      in order to practice charity."     Albert Camus
>>
>>
>
>
> ----------------------------------------------------------------------- 
> -------
> Notice:  This e-mail message, together with any  
> attachments,...{{dropped}}
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!  
> http://www.R-project.org/posting-guide.html