[R] using ddply with segmented regression
Prew, Paul
Paul.Prew at ecolab.com
Tue Oct 15 01:23:04 CEST 2013
Yes, David, sorry for the confusion. I forgot arun had proposed a genericized solution, where the dataframe name was shortened from "Test50.df" to "df". I could have replaced arun's 'df' references to 'Test50.df', but lazily changed my "Test50.df" to "df" and used arun's code verbatim.
The unmatched parenthesis you caught (cut&paste error) is fixed here: SP.seg <- dlply(df,.(Lot.Run),segmentf_df)
>> SP.out <- ldply(SP.seg)
As for your (excerpted) question --
*So what function were you intending to be used in that call to ldply ... If you are using ldply to process the models with plot.segmented, .. object returned will be an empty dataframe but the plots will be done.*
I wanted to look at the data frame first so I could understand the model output represented in row x column format. Then I planned to try
> ldply(SP.seg, segmented.plot)
If the above printed out the 22 piecewise regressions individually, then the next step would be determining how they could be arranged in a 1-page lattice. I think the chemist who generated the results would profit from that. The 1-pager would require me investigating lattice or ggplot2.
Paul Prew | Statistician
651-795-5942 | fax 651-204-7504
Ecolab Research Center | Mail Stop ESC-F4412-A
655 Lone Oak Drive | Eagan, MN 55121-1560
-----Original Message-----
From: David Winsemius [mailto:dwinsemius at comcast.net]
Sent: Monday, October 14, 2013 5:30 PM
To: Prew, Paul
Cc: arun; R help
Subject: Re: [R] using ddply with segmented regression
On Oct 14, 2013, at 2:57 PM, Prew, Paul wrote:
> Hello, the code provided by arun did the trick. Thank you very much, arun.
>
> However, I'm now unsure of how to further process the results .
> Looking at the vignette aka "split-apply-combine". It appears that I
> could now create a dataframe from the list of results, and then run
> the results through the function plot.segmented to view the piecewise
> regressions by the grouping variable Lot.Run. However, the list is
> not in the structure expected by ldply --
>
>> SP.seg <- dlply(df,.(Lot.Run),segmentf_df)
That wasn't the name of the dataframe you offered in the first post.... and this code could not possibly have not thrown an error since there are unmatched parens.
>> SP.out <- ldply(SP.seg)
So what function were you intending to be used in that call to ldply ... after you fix the errors above? If you are using ldply to process the models with plot.segmented, then realize that the object returned will be an empty dataframe but hte plots will be done.
>
> [9] ERROR:
> Results must be all atomic, or all data frames
>
>> class(SP.seg)[[1]]
> [1] "list"
>
>> head(SP.seg)
> $`J062431-1`
> Call: segmented.lm(obj = out.lm, seg.Z = ~Cycle, psi = (Cycle = NA),
> control = seg.control(stop.if.error = FALSE, n.boot = 0,
> gap = FALSE, jt = FALSE, nonParam = TRUE))
>
> Meaningful coefficients of the linear terms:
> (Intercept) Cycle U1.Cycle U2.Cycle U3.Cycle U4.Cycle U5.Cycle U6.Cycle
> 40.11786 -0.06664 -0.68539 0.49316 0.14955 0.03612 0.22257 -0.41166
> U7.Cycle U8.Cycle U9.Cycle U10.Cycle
> -0.48365 0.37949 0.24945 0.06712
>
> Estimated Break-Point(s) psi1.Cycle psi2.Cycle psi3.Cycle psi4.Cycle
> psi5.Cycle psi6.Cycle psi7.Cycle psi8.Cycle psi9.Cycle psi10.Cycle :
> 19.67 34.31 51.02 72.10 97.94 117.20 130.10 147.10 155.70 160.40
>
> $`J062431-2`
> Call: segmented.lm(obj = out.lm, seg.Z = ~Cycle, psi = (Cycle = NA),
> control = seg.control(stop.if.error = FALSE, n.boot = 0,
> gap = FALSE, jt = FALSE, nonParam = TRUE))
>
> Meaningful coefficients of the linear terms:
> (Intercept) Cycle U1.Cycle U2.Cycle U3.Cycle U4.Cycle U5.Cycle U6.Cycle
> 40.11786 -0.06664 -0.68539 0.49316 0.14955 0.03612 0.22257 -0.41166
> U7.Cycle U8.Cycle U9.Cycle U10.Cycle
> -0.48365 0.37949 0.24945 0.06712
>
> Estimated Break-Point(s) psi1.Cycle psi2.Cycle psi3.Cycle psi4.Cycle
> psi5.Cycle psi6.Cycle psi7.Cycle psi8.Cycle psi9.Cycle psi10.Cycle :
> 19.67 34.31 51.02 72.10 97.94 117.20 130.10 147.10 155.70 160.40
>
> My hope was to eventually increase my understanding enough to create lattice plots using 'segment.plot' via ldply. Will that even work with the output object from this segmented package?
Hard to tell. You seem to be changing the names of your objects at random.
>
> Thanks,Paul
>
> Paul Prew | Statistician
> 651-795-5942 | fax 651-204-7504
> Ecolab Research Center | Mail Stop ESC-F4412-A
> 655 Lone Oak Drive | Eagan, MN 55121-1560
>
> -----Original Message-----
> From: arun [mailto:smartpink111 at yahoo.com]
> Sent: Saturday, October 12, 2013 1:42 AM
> To: R help
> Cc: Prew, Paul
> Subject: Re: [R] using ddply with segmented regression
>
>
>
> Hi,
> Try:
>
> segmentf_df <- function(df) {
> out.lm<-lm(deltaWgt~Cycle, data=df)
> segmented(out.lm,seg.Z=~Cycle,
> psi=(Cycle=NA),control=seg.control(stop.if.error=FALSE,n.boot=0))
> }
>
> library(plyr)
> library(segmented)
>
> dlply(df,.(Lot.Run),segmentf_df)
> $`J062431-1`
> Call: segmented.lm(obj = out.lm, seg.Z = ~Cycle, psi = (Cycle = NA),
> control = seg.control(stop.if.error = FALSE, n.boot = 0))
>
> Meaningful coefficients of the linear terms:
> (Intercept) Cycle U1.Cycle U2.Cycle
> 38.480 1.130 -2.760 1.497
>
> Estimated Break-Point(s) psi1.Cycle psi2.Cycle : 3.732 5.056
>
> $`J062431-2`
> Call: segmented.lm(obj = out.lm, seg.Z = ~Cycle, psi = (Cycle = NA),
> control = seg.control(stop.if.error = FALSE, n.boot = 0))
>
> Meaningful coefficients of the linear terms:
> (Intercept) Cycle U1.Cycle U2.Cycle
> 48.4300 -3.2500 3.0905 -0.6555
>
> Estimated Break-Point(s) psi1.Cycle psi2.Cycle : 2.12 22.15
>
> attr(,"split_type")
> [1] "data.frame"
> attr(,"split_labels")
> Lot.Run
> 1 J062431-1
> 2 J062431-2
>
>
> #or
>
> dlply(df,.(Lot.Run),function(x) segmentf_df(x)) #or
> lapply(split(df,df$Lot.Run,drop=TRUE),function(x) segmentf_df(x))
>
>
> A.K.
>
>
> On Friday, October 11, 2013 11:16 PM, "Prew, Paul" <Paul.Prew at ecolab.com> wrote:
> Hello,
>
> I'm unsuccessfully trying to apply piecewise linear regression over each of 22 groups. The data structure of the reproducible toy dataset is below. I'm using the 'segmented' package, it worked fine with a data set that containing only one group ("Lot.Run").
>
> $ Cycle : int 1 2 3 4 5 6 7 8 9 10 ...
> $ Lot.Run : Factor w/ 22 levels "J062431-1","J062431-2",..: 1 1 1 1 1 1 1 1 1 1 ...
> $ deltaWgt: num 38.7 42.6 41 42.3 40.6 ...
>
> I am new to 'segmented', and also new to 'plyr', which is how I'm trying to apply this segmented regression to the 22 Lot.Run groups. Within a Lot.Run, the piecewise linear regressions are deltaWgt vs. Cycle.
>
> ##### define the linear regression ##### out.lm<-lm(deltaWgt~Cycle,
> data=Test50.df)
>
> ##### define the function called by dlply #####
> ##### find cutpoints via bootstrapping, fit the piecewise
> regressions ##### segmentf_df <- function(df) {
> segmented(out.lm,seg.Z=~Cycle,
> psi=(Cycle=NA),control=seg.control(stop.if.error=FALSE,n.boot=0)),
> data = df) }
>
> at this point, there's an error message 23] ERROR: <text>
>
> ##### repeat for each Lot.Run group #####
> dlply(Test50.df, .(Lot.Run), segmentf_df)
>
> at this point, there's an error message [28] ERROR:
> object 'segmentf_df' not found
>
> Any suggestions?
> Thanks, Paul
>
>> dput(Test50.df)
> structure(list(Cycle = c(1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L,
> 12L, 13L, 14L, 15L, 16L, 17L, 18L, 19L, 20L, 21L, 22L, 23L, 24L, 25L,
> 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 16L,
> 17L, 18L, 19L, 20L, 21L, 22L, 23L, 24L, 25L), Lot.Run =
> structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,
> 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L,
> 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L,
> 2L, 2L), .Label = c("J062431-1", "J062431-2", "J062431-3",
> "J062432-1", "J062432-2", "J062433-1", "J062433-2", "J062433-3", "Lot
> 1-1", "Lot 1-2", "Lot 2-1", "Lot 2-2", "Lot 2-3", "Lot 3-1", "Lot
> 3-2", "Lot 3-3", "P041231-1", "P041231-2", "P041531-1", "P041531-2",
> "P041531-3", "P041531-4"), class = "factor"),
> deltaWgt = c(38.69, 42.58, 40.95, 42.26, 40.63, 41.61, 36.73,
> 41.28, 39.98, 40.63, 39.66, 39.98, 40.95, 38.36, 39.01, 39,
> 38.03, 39.66, 37.7, 39.66, 40.63, 38.03, 37.71, 36.73, 37.7,
> 45.18, 41.93, 42.59, 39.98, 40.95, 42.91, 38.03, 40.96, 39,
> 41.61, 39.33, 43.88, 39.98, 38.68, 38.68, 36.08, 39.99, 38.35,
> 40.31, 40.63, 38.68, 37.05, 38.36, 35.43, 36.73)), .Names =
> c("Cycle", "Lot.Run", "deltaWgt"), row.names = c(1L, 2L, 3L, 4L, 5L,
> 6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 16L, 17L, 18L, 19L, 20L,
> 21L, 22L, 23L, 24L, 25L, 207L, 208L, 209L, 210L, 211L, 212L, 213L,
> 214L, 215L, 216L, 217L, 218L, 219L, 220L, 221L, 222L, 223L, 224L,
> 225L, 226L, 227L, 228L, 229L, 230L, 231L), class = "data.frame")
>
>
>
David Winsemius
Alameda, CA, USA
CONFIDENTIALITY NOTICE: This e-mail communication and an...{{dropped:7}}
More information about the R-help
mailing list