[R] need help with smooth.spline
W. C. Thacker
Carlisle.Thacker at noaa.gov
Thu Mar 4 21:03:50 CET 2004
Dear Prof. Ripley,
I'm not sure what you are asking.
Prof Brian Ripley wrote:
>
> Can we distinguish
>
> smooth.spline() with a fixed penalty lambda.
>
> smooth.spline() with a particular smoothing selector, e.g. GCV (the
> default).
>
> I doubt if the smoothing fails but the selection of lambda might if the
> true curve is not smooth.
I have used the default GCV to determine how much smoothing. This
generally works fine, and for the cases where it has difficulties it
still works well for most of the data.
My original posting had a few examples in an attachment that was
stripped off but later reposted. Andy Liaw has verified (see below)
that smooth.spline() works well with these data at the original
points, and I have suggested pictures showing how it fails away from
the original points. They do not show the extreme magnitudes of the
bad values, as they lie beyond the range of the pictures
Should I be using something other than the defaults? I need something
that will work for many cases.
Is there a way to recognize which data are likely to cause problems
for smooth.spline(), which might be used to separate them for special
treatment?
Thanks,
Carlisle
> Andy,
>
> You are seeing that smooth.spline() reproduces the observations. The
> problem is with predicting away from the observations.
>
> > I tried looking at the data you posted but don't see anything wrong with
> > smooth.spline(). Am I looking at the right thing?
> >
> > > dat <- read.table("clipboard", header=T) # Read data from the Windoze
> > clipboard
> > > dat <- split(x[,2:4], x$id)
> > > str(dat)
> > List of 4
> > $ 3281507:`data.frame': 77 obs. of 3 variables:
> > ..$ p: num [1:77] 0 44 47.9 68.9 69.9 71.9 72.9 73.9 74.9 75.9 ...
> > ..$ t: num [1:77] 27.5 27.6 27.6 27.5 27.5 ...
> > ..$ s: num [1:77] 35.7 35.7 35.7 35.7 35.8 ...
> > $ 3322643:`data.frame': 70 obs. of 3 variables:
> > ..$ p: num [1:70] 0 42 44 47 49 53 55 58 59 63 ...
> > ..$ t: num [1:70] 23.3 23.3 23.2 22.1 21.3 ...
> > ..$ s: num [1:70] 35.5 35.5 35.5 35.6 35.6 ...
> > $ 3336561:`data.frame': 62 obs. of 3 variables:
> > ..$ p: num [1:62] 0 37 39 40 43 47 49 50 52 53 ...
> > ..$ t: num [1:62] 24.5 24.5 24.4 24.4 23.8 ...
> > ..$ s: num [1:62] 35.3 35.3 35.3 35.4 35.5 ...
> > $ 3337798:`data.frame': 59 obs. of 3 variables:
> > ..$ p: num [1:59] 0 4 11 15 43 44 45 47 50 57 ...
> > ..$ t: num [1:59] 27.5 27.4 27.4 27.3 27.2 ...
> > ..$ s: num [1:59] 34.6 34.6 34.6 34.6 34.7 ...
> > > par(mfrow=c(2,2))
> > > for(i in 1:4) { plot(dat[[i]]$p, dat[[i]]$s);
> > lines(smooth.spline(dat[[i]]$p, dat[[i]]$s))}
>
> Look at this:
>
> par(mfrow=c(2,4))
> for(i in 1:4) {
> plot(dat[[i]]$p, dat[[i]]$t);
> lines(seq(25,1000,25),
> predict(smooth.spline(dat[[i]]$p,
> dat[[i]]$t),seq(25,1000,25))$y);
> plot(dat[[i]]$p, dat[[i]]$s);
> lines(seq(25,1000,25),
> predict(smooth.spline(dat[[i]]$p,
> dat[[i]]$s),seq(25,1000,25))$y)}
>
> or at this:
>
> par(mfrow=c(2,4))
> for(i in 1:4) {
> plot(dat[[i]]$p, dat[[i]]$t);
> lines(seq(25,1000,1),
> predict(smooth.spline(dat[[i]]$p,
> dat[[i]]$t),seq(25,1000,1))$y);
> plot(dat[[i]]$p, dat[[i]]$s);
> lines(seq(25,1000,1),
> predict(smooth.spline(dat[[i]]$p,
> dat[[i]]$s),seq(25,1000,1))$y)}
>
> Thanks,
>
> Carlisle
id p t s
3322643 0.0 23.26 35.490
3322643 42.0 23.26 35.500
3322643 44.0 23.19 35.550
3322643 47.0 22.12 35.600
3322643 49.0 21.34 35.639
3322643 53.0 17.34 35.610
3322643 55.0 16.23 35.580
3322643 58.0 15.51 35.520
3322643 59.0 15.39 35.460
3322643 63.0 14.82 35.450
3322643 67.0 14.67 35.470
3322643 68.0 14.66 35.480
3322643 72.0 14.37 35.450
3322643 76.0 14.26 35.420
3322643 82.0 13.92 35.380
3322643 86.0 13.83 35.370
3322643 94.0 13.49 35.350
3322643 104.0 13.23 35.340
3322643 112.0 13.09 35.310
3322643 127.0 12.76 35.270
3322643 134.0 12.71 35.270
3322643 138.0 12.63 35.270
3322643 142.0 12.60 35.260
3322643 146.0 12.47 35.240
3322643 155.0 12.39 35.240
3322643 164.0 12.34 35.230
3322643 170.0 12.18 35.210
3322643 172.0 12.16 35.210
3322643 175.0 12.09 35.210
3322643 201.0 11.74 35.160
3322643 206.0 11.67 35.150
3322643 218.0 11.57 35.130
3322643 223.0 11.47 35.130
3322643 232.0 11.38 35.120
3322643 243.0 11.29 35.110
3322643 253.0 11.16 35.100
3322643 265.0 11.10 35.100
3322643 277.0 11.10 35.110
3322643 285.0 10.98 35.100
3322643 295.0 10.87 35.090
3322643 305.0 10.78 35.090
3322643 320.0 10.68 35.090
3322643 330.0 10.58 35.080
3322643 340.0 10.62 35.090
3322643 355.0 10.53 35.090
3322643 375.0 10.55 35.150
3322643 395.0 10.46 35.140
3322643 405.0 10.36 35.140
3322643 425.0 10.25 35.150
3322643 470.0 9.80 35.120
3322643 480.0 9.78 35.120
3322643 515.0 9.39 35.090
3322643 540.0 8.95 35.060
3322643 545.0 8.94 35.050
3322643 565.0 8.66 35.030
3322643 570.0 8.64 35.020
3322643 610.0 8.13 34.970
3322643 630.0 7.99 34.950
3322643 660.0 7.70 34.930
3322643 695.0 7.20 34.900
3322643 745.0 6.62 34.840
3322643 780.0 6.46 34.820
3322643 795.0 6.29 34.820
3322643 810.0 6.26 34.820
3322643 860.0 5.91 34.800
3322643 905.0 5.72 34.810
3322643 920.0 5.67 34.820
3322643 940.0 5.64 34.830
3322643 975.0 5.57 34.840
3322643 985.0 5.57 34.850
3336561 0.0 24.48 35.330
3336561 37.0 24.46 35.340
3336561 39.0 24.43 35.340
3336561 40.0 24.40 35.400
3336561 43.0 23.81 35.500
3336561 47.0 20.90 35.749
3336561 49.0 17.27 35.829
3336561 50.0 16.60 35.799
3336561 52.0 16.18 35.709
3336561 53.0 16.11 35.699
3336561 56.0 15.83 35.590
3336561 60.0 15.14 35.550
3336561 70.0 14.76 35.530
3336561 71.0 14.73 35.520
3336561 73.0 14.62 35.510
3336561 75.0 14.52 35.500
3336561 85.0 14.38 35.480
3336561 106.0 14.29 35.460
3336561 109.0 14.19 35.450
3336561 112.0 14.09 35.420
3336561 113.0 14.09 35.430
3336561 118.0 13.83 35.400
3336561 127.0 13.75 35.380
3336561 137.0 13.58 35.370
3336561 148.0 13.49 35.360
3336561 166.0 13.43 35.350
3336561 179.0 13.38 35.340
3336561 192.0 13.25 35.300
3336561 197.0 13.08 35.300
3336561 214.0 12.87 35.270
3336561 223.0 12.77 35.250
3336561 229.0 12.44 35.210
3336561 235.0 12.02 35.140
3336561 237.0 11.79 35.130
3336561 239.0 11.53 35.100
3336561 242.0 11.34 35.100
3336561 247.0 11.21 35.090
3336561 248.0 11.19 35.040
3336561 252.0 10.87 35.060
3336561 256.0 10.71 34.980
3336561 258.0 10.63 35.040
3336561 269.0 10.51 34.990
3336561 276.0 10.32 34.980
3336561 287.0 10.25 34.980
3336561 294.0 10.07 34.960
3336561 298.0 9.86 34.920
3336561 300.0 9.80 34.920
3336561 310.0 9.76 34.900
3336561 320.0 9.48 34.890
3336561 325.0 9.46 34.890
3336561 350.0 9.07 34.860
3336561 355.0 8.93 34.840
3336561 375.0 8.53 34.780
3336561 380.0 8.33 34.770
3336561 390.0 8.09 34.750
3336561 400.0 8.00 34.730
3336561 410.0 7.65 34.700
3336561 430.0 7.29 34.650
3336561 435.0 7.21 34.640
3336561 470.0 6.93 34.620
3336561 495.0 6.78 34.600
3336561 510.0 6.70 34.590
3337798 0.0 27.46 34.600
3337798 4.0 27.44 34.630
3337798 11.0 27.44 34.640
3337798 15.0 27.28 34.640
3337798 43.0 27.16 34.680
3337798 44.0 27.13 34.820
3337798 45.0 26.80 34.970
3337798 47.0 24.45 35.300
3337798 50.0 22.53 35.430
3337798 57.0 18.07 35.460
3337798 59.0 16.97 35.580
3337798 60.0 16.69 35.580
3337798 62.0 16.23 35.610
3337798 65.0 15.89 35.550
3337798 73.0 14.63 35.500
3337798 77.0 14.44 35.470
3337798 84.0 13.97 35.450
3337798 90.0 13.79 35.450
3337798 94.0 13.69 35.410
3337798 97.0 13.65 35.410
3337798 120.0 13.54 35.400
3337798 144.0 13.34 35.370
3337798 164.0 13.20 35.340
3337798 177.0 13.09 35.340
3337798 181.0 12.97 35.340
3337798 183.0 12.93 35.320
3337798 201.0 12.63 35.270
3337798 203.0 12.61 35.280
3337798 216.0 12.46 35.240
3337798 221.0 12.35 35.200
3337798 224.0 12.15 35.200
3337798 226.0 11.97 35.100
3337798 231.0 11.57 35.140
3337798 233.0 11.55 35.140
3337798 234.0 11.50 35.090
3337798 240.0 11.12 35.020
3337798 244.0 10.84 35.040
3337798 248.0 10.81 35.020
3337798 253.0 10.62 34.990
3337798 256.0 10.25 34.960
3337798 257.0 10.18 34.970
3337798 265.0 10.09 34.950
3337798 271.0 9.93 34.940
3337798 274.0 9.91 34.930
3337798 279.0 9.54 34.880
3337798 283.0 9.38 34.880
3337798 287.0 9.31 34.850
3337798 292.0 9.03 34.840
3337798 301.0 8.79 34.830
3337798 310.0 8.74 34.820
3337798 350.0 7.81 34.740
3337798 355.0 7.73 34.720
3337798 400.0 7.55 34.700
3337798 415.0 7.52 34.690
3337798 445.0 7.25 34.660
3337798 460.0 6.99 34.640
3337798 480.0 6.78 34.610
3337798 505.0 6.32 34.580
3337798 515.0 6.30 34.580
3281507 0.0 27.52 35.694
3281507 44.0 27.55 35.694
3281507 47.9 27.55 35.694
3281507 68.9 27.54 35.724
3281507 69.9 27.50 35.764
3281507 71.9 26.67 36.014
3281507 72.9 26.36 36.054
3281507 73.9 25.97 36.034
3281507 74.9 25.24 36.044
3281507 75.9 24.54 36.094
3281507 78.9 24.04 36.124
3281507 79.9 23.76 36.114
3281507 80.9 23.67 36.104
3281507 84.9 23.13 36.084
3281507 86.9 22.74 36.054
3281507 87.9 22.62 36.064
3281507 89.9 22.28 36.034
3281507 90.9 22.18 36.044
3281507 91.9 22.08 36.044
3281507 92.9 21.76 35.974
3281507 93.9 21.00 35.794
3281507 94.8 20.40 35.754
3281507 95.8 19.08 35.544
3281507 96.8 18.39 35.594
3281507 97.8 17.97 35.534
3281507 99.8 16.86 35.594
3281507 101.8 16.65 35.634
3281507 104.8 16.28 35.624
3281507 105.8 16.22 35.634
3281507 107.8 16.17 35.644
3281507 108.8 16.15 35.634
3281507 115.8 15.75 35.604
3281507 119.7 15.56 35.604
3281507 127.7 15.40 35.584
3281507 134.7 15.24 35.564
3281507 141.6 15.14 35.554
3281507 147.6 14.98 35.534
3281507 158.6 14.79 35.514
3281507 165.5 14.71 35.514
3281507 174.5 14.58 35.494
3281507 190.4 14.46 35.474
3281507 209.3 14.13 35.434
3281507 218.2 13.99 35.414
3281507 228.2 13.82 35.394
3281507 235.2 13.72 35.384
3281507 246.1 13.59 35.364
3281507 256.0 13.49 35.354
3281507 269.0 13.36 35.334
3281507 276.9 13.32 35.334
3281507 284.9 13.21 35.314
3281507 297.8 12.88 35.274
3281507 311.7 12.71 35.254
3281507 325.6 12.42 35.234
3281507 342.5 11.82 35.154
3281507 351.5 11.25 35.094
3281507 357.4 11.09 35.084
3281507 362.4 11.05 35.084
3281507 367.4 10.80 35.064
3281507 371.4 10.73 35.054
3281507 379.3 10.40 35.034
3281507 392.2 10.38 35.044
3281507 402.2 10.12 35.014
3281507 405.1 9.98 34.994
3281507 412.1 9.69 34.964
3281507 415.1 9.44 34.934
3281507 427.0 8.98 34.884
3281507 431.0 8.87 34.874
3281507 437.9 8.68 34.854
3281507 441.9 8.61 34.844
3281507 446.9 8.49 34.834
3281507 455.8 8.28 34.804
3281507 468.7 8.11 34.784
3281507 473.7 8.07 34.774
3281507 484.6 7.87 34.754
3281507 495.5 7.74 34.754
3281507 501.5 7.68 34.734
3281507 508.4 7.59 34.734
--
William Carlisle Thacker
Atlantic Oceanographic and Meteorological Laboratory
4301 Rickenbacker Causeway, Miami, Florida 33149 USA
Office: (305) 361-4323 Fax: (305) 361-4392
"Too many have dispensed with generosity
in order to practice charity." Albert Camus
More information about the R-help
mailing list