[R] complicated time series filtering issue

Ebert,Timothy Aaron tebert @end|ng |rom u||@edu
Mon Apr 4 23:30:42 CEST 2022


I think the idea is more 
for (i in 2:nrow(x)){
ifelse(x[i]-x[i-1] >10) {keep x[i], delete x[i]]
}

I am not quite clear on the correct code for "keep" or "delete."

One could try 
for (i in 2:nrow(x)){
x$new[i] <- x[i]-x[i-1]
}
x <- x %>% filter(new>=10)

This only works if consecutive sample dates are 10 or more days apart. You could add an else if that would accumulate days, and if successful reset the clock.

Tim
-----Original Message-----
From: R-help <r-help-bounces using r-project.org> On Behalf Of Bert Gunter
Sent: Monday, April 4, 2022 5:04 PM
To: Cade, Brian S <cadeb using usgs.gov>
Cc: r-help using r-project.org
Subject: Re: [R] complicated time series filtering issue

[External Email]

Like this?

winnow <- function(x, int=5){
   keep <- x[1]
   remaining <- x[-1]
   while (length(remaining))
   {
      nxt <- tail(keep,1) + int
      if(length(remaining) ==1 ||
         all(remaining < nxt))break
      remaining <- remaining[remaining >tail(keep,1) + int]
      keep <- c(keep,remaining[1])
   }
   keep
}

> x
 [1]  1  2  5  7  8  9 15 16 17 19 20 21 28 35 37 41 43 45 46 50
> winnow(x,7)
[1]  1  9 17 28 37 45
> winnow(x,5)
[1]  1  7 15 21 28 35 41 50

Cheers,
Bert

"The trouble with having an open mind is that people keep coming along and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )

On Mon, Apr 4, 2022 at 12:56 PM Cade, Brian S via R-help <r-help using r-project.org> wrote:
>
> Hello:  I have an issue with filtering in a time series of animal 
> growth data that seems conceptually simple but I have not come up with 
> effective code to implement this.  I have temporal sequences of 
> lengths by individuals and I want to retain only those data that are 
> >10 days apart sequentially within an individuals records.  I can 
> readily compute intervals between successive dates by individual using 
> data.table() and its by = INDIVIDUAL functionality.  See example data 
> for one individual below.  But what currently eludes me in processing 
> this is how to recognize for example that deleting the 2nd and 3rd 
> rows is required because the totality of their time interval is 9 
> days, deleting 8th record with 4 days is required,  deleting 17th 
> record with 1 day is required, deleting 22nd and 23rd records is 
> required because their sum is 2 days, but we do not delete 24th record 
> of 10 days because the sum of previous 2 records deleted and this one 
> is now 12 days.  Each individual can have ve
 ry
>   different patterns of these sort of sequences.  These sequences are easy to look at and determine what needs to be done but writing effective code to accomplish this filtering seems to require some functionality that I am currently missing.
>
> Any suggestions would be greatly appreciated.
>
>          Date   INDIVIDUAL DATENUMBER LENGTH length.prev interval
> 228 12-May-04 57084544        133         682.4           NA       NA
> 229 28-Sep-04 57084544        272         724.8        682.4      139
> 230 30-Sep-04 57084544        274         740.8        724.8        2
> 231  7-Oct-04 57084544        281         745.4        740.8        7
> 232 22-Nov-04 57084544        327         780.2        745.4       46
> 233 27-Jan-05 57084544        393         817.2        780.2       66
> 234  8-Mar-05 57084544        433         834.1        817.2       40
> 235  2-Jul-05 57084544        549         876.3        834.1      116
> 236  6-Jul-05 57084544        553         871.5        876.3        4
> 237  4-Aug-05 57084544        582         887.5        871.5       29
> 238 28-Dec-05 57084544        728         921.8        887.5      146
> 239 31-Jan-06 57084544        762         936.8        921.8       34
> 240 27-Feb-06 57084544        789         962.4        936.8       27
> 241 21-Nov-06 57084544       1056         972.3        962.4      267
> 242 30-Mar-07 57084544       1185        1007.2        972.3      129
> 243 23-Apr-07 57084544       1209        1009.1       1007.2       24
> 244 22-May-07 57084544       1238         991.6       1009.1       29
> 245 23-May-07 57084544       1239        1015.9        991.6        1
> 246 16-Jul-07 57084544       1293        1006.5       1015.9       54
> 247  9-Aug-07 57084544       1317        1013.0       1006.5       24
> 248 27-Aug-07 57084544       1335        1013.0       1013.0       18
> 249 29-Jul-08 57084544       1672        1021.5       1013.0      337
> 250 30-Jul-08 57084544       1673         984.3       1021.5        1
> 251 31-Jul-08 57084544       1674        1008.5        984.3        1
> 252 10-Aug-08 57084544       1684        1002.8       1008.5       10
> 253 22-Oct-08 57084544       1757         977.6       1002.8       73
> 254  2-Dec-08 57084544       1798        1000.6        977.6       41
>
>
>
> Brian
>
>
>
> Brian S. Cade, PhD
>
> U. S. Geological Survey
> Fort Collins Science Center
> 2150 Centre Ave., Bldg. C
> Fort Collins, CO  80526-8818
>
> email:  cadeb using usgs.gov<mailto:brian_cade using usgs.gov>
> tel:  970 226-9326
>
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mail
> man_listinfo_r-2Dhelp&d=DwICAg&c=sJ6xIWYx-zLMB3EPkvcnVg&r=9PEhQh2kVeAs
> Rzsn7AkP-g&m=ZfVdnGSALzyajo_d1U09NJs3RCXcx5NwQ2PZ9A9zwEnVYnexn4toTyxgu
> -vCEJab&s=PG1chCZY6eQzSdtSlvChVVVt0HXVDG1bgBkJMQ8wk1A&e=
> PLEASE do read the posting guide 
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.R-2Dproject.or
> g_posting-2Dguide.html&d=DwICAg&c=sJ6xIWYx-zLMB3EPkvcnVg&r=9PEhQh2kVeA
> sRzsn7AkP-g&m=ZfVdnGSALzyajo_d1U09NJs3RCXcx5NwQ2PZ9A9zwEnVYnexn4toTyxg
> u-vCEJab&s=D_bzOVjWanUgYD_zJq-IS8EObMKBmC5Q5D-a_IHxMAA&e=
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Dhelp&d=DwICAg&c=sJ6xIWYx-zLMB3EPkvcnVg&r=9PEhQh2kVeAsRzsn7AkP-g&m=ZfVdnGSALzyajo_d1U09NJs3RCXcx5NwQ2PZ9A9zwEnVYnexn4toTyxgu-vCEJab&s=PG1chCZY6eQzSdtSlvChVVVt0HXVDG1bgBkJMQ8wk1A&e=
PLEASE do read the posting guide https://urldefense.proofpoint.com/v2/url?u=http-3A__www.R-2Dproject.org_posting-2Dguide.html&d=DwICAg&c=sJ6xIWYx-zLMB3EPkvcnVg&r=9PEhQh2kVeAsRzsn7AkP-g&m=ZfVdnGSALzyajo_d1U09NJs3RCXcx5NwQ2PZ9A9zwEnVYnexn4toTyxgu-vCEJab&s=D_bzOVjWanUgYD_zJq-IS8EObMKBmC5Q5D-a_IHxMAA&e=
and provide commented, minimal, self-contained, reproducible code.



More information about the R-help mailing list