[R] R - Aggregate 3-Hourly Block Data into Weekly (Melt)

[Jeff Newmiller]

Thank you for attempting to convey your problem clearly using example 
code... but your use of HTML email has very nearly undone all your 
efforts.  Also, use of "dput" to make an R-readable block of data is more 
reliable than read.table to get the data into our R sessions quickly.

First question: you say these are three-hour results, but there are only 
six per day in your example.

Second question: you say the "number of blocks vary between 7 and 8", but 
your example data does not illustrate that problem. (If there were 8 
blocks per day the three-hour statement would make more sense.)

Comment: You have not mentioned timezone information... this information 
looks ripe for GMT, but if that is a bad assumption then daylight savings 
might account for some variations in blocks per day.

Comment: I think your plan of using rep to identify the week numbers is 
risky. I would recommend using a Date or POSIXt type to make the 
timestamps computable, and then find the date corresponding to the 
beginning of the week that the timestamp falls into. Then aggregate 
grouping on those time values. Unfortunately, the specific way you go 
about identifying the beginning of week may depend on the timezone 
information.

On Thu, 18 Dec 2014, Shouro Dasgupta wrote:

> I am trying to compute max, min, and mean from Global Circulation Models
> (GCM) for the US. The data is in 3-hour blocks for 2026-2045 and 2081-2100.
> Sample Data:
>
> tmp1 <- structure(list(FIPS = c(1001L, 1003L, 1005L), X2026.01.01.1 =
> c(285.5533142,
>  285.5533142, 286.2481079), X2026.01.01.2 = c(283.4977112, 283.4977112,
>  285.0860291), X2026.01.01.3 = c(281.9733887, 281.9733887, 284.1548767
>  ), X2026.01.01.4 = c(280.0234985, 280.0234985, 282.6075745),
>      X2026.01.01.5 = c(278.7125854, 278.7125854, 281.2553711),
>      X2026.01.01.6 = c(278.5204773, 278.5204773, 280.6148071)),
> .Names = c("FIPS",
>  "X2026.01.01.1", "X2026.01.01.2", "X2026.01.01.3", "X2026.01.01.4",
>  "X2026.01.01.5", "X2026.01.01.6"), class = "data.frame", row.names = c(NA,
>  -3L))
>
> I have extracted the data by FIPS code and reshaped the yearly data files
> using melt();
>
> for (i in filelist) {
>  tmp1 <- as.data.table(read.csv(i,header=T, sep=","))
>  tmp2 <- melt(tmp1, id="FIPS")
>  tmp2$year <- as.numeric(substr(tmp2$variable,2,5))
>  tmp2$month <- as.numeric(substr(tmp2$variable,7,8))
>  tmp2$day <- as.numeric(substr(tmp2$variable,10,11))}
>
> I have added datestring and weekdays using the following code:
> Inserting Date Variable
>
> tmp2$date <- with(tmp2, ymd(sprintf('%04d%02d%02d', year, month, day)))
>
> Inserting Day Variable
>
> tmp2$day <- weekdays(as.Date(tmp2$date))
>
> sample.tmp2 <- "FIPS         xdate     temp year month      day
> date      dates weekdays
> + 5599311  1003 X2045.08.14.2 304.5995 2045     8   Monday 2045-08-14
> 2036-01-29        2
> + 468406  39093 X2045.01.19.7 267.8483 2045     1 Thursday 2045-01-19
> 2028-06-04        0
> + 5022078 21167 X2045.07.21.8 314.6772 2045     7   Friday 2045-07-21
> 2035-09-13        4
> + 186822   9005 X2045.01.08.5 269.0803 2045     1   Sunday 2045-01-08
> 2037-06-28        0
> + 3998678 13295 X2045.06.10.7 307.2408 2045     6 Saturday 2045-06-10
> 2033-10-13        4"
>
> Data <- read.table(text=sample.tmp2, header = TRUE)
>
> My goal is to aggregate these 3-hourly blocks into weekly data, however,
> GCM data is not consistent and the blocks vary between 7 and 8. I want to
> clip the data to start on the first Monday of 2026 and end on the last
> Sunday of 2045 and then use rep() to assign week numbers for the whole
> epoch.
>
> I know I can count the number of each day using something like this;
>
> length(which(weekdays == '0'))
>
> Where 0, 1, 2..., 6 represent Sunday, Monday,...
>
> My question is am I doing anything wrong in trying to aggregate the data to
> begin with? But importantly, I would be grateful for any help to clip the
> dataset to begin on the first Monday and end on the last Sunday. Thank you
> very much!
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

---------------------------------------------------------------------------
Jeff Newmiller                        The     .....       .....  Go Live...
DCN:<jdnewmil at dcn.davis.ca.us>        Basics: ##.#.       ##.#.  Live Go...
                                       Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/Batteries            O.O#.       #.O#.  with
/Software/Embedded Controllers)               .OO#.       .OO#.  rocks...1k