Dear R-help,

I am working with some data stored as "filename.txt.gz" in my working
directory.
After reading the data in using read.table(), I can see that each of them
has four columns (variable, id, outcome, and rate) and the following
structure:

# sample data
x2 <- data.frame(variable = rep(paste0('x', 1:100), each = 100), id =
rep(paste0('p', 1:100), 100), outcome = sample(0:2, 10000, TRUE), rate =
runif(10000, 0.5, 1))
str(x2)

Each variable, i.e., x1, x2,..., x100 is repeated as many times as the
number of unique IDs (100 in this example).  What I would like to do is to
transform the data above
in a long format.  I can do this by using

# reshape
wide <- reshape(x2[, -4], v.names = "outcome", idvar = "id",
                timevar = "variable", direction = "wide")
str(wide)

# or a "hack" with mclapply:

require(parallel)
sel <- as.character(unique(x2$variable))
id <- as.character(unique(x2$id))
X <- matrix(NA, ncol = length(sel) + 1, nrow = length(id))
X[, 1] <- id
colnames(X) <- c('id', sel)
r <- mclapply(seq_along(sel), function(i){
                        out <- x2[x2$variable == sel[i], ][, 3]
                        }, mc.cores = 4)
X[, -1] <- do.call(rbind, r)
X

However, I was wondering if it is possible to come up with another solution
, hopefully faster than these
.  Unfortunately, either one of these takes a very long time to process,
specially when the number of variables is very large
(> 250,000) and the number of ids is ~2000.

I would very much appreciate your suggestions.   At the end of this message
is my sessionInfo().

Thank you very much in advance.

Best regards,
Jorge Velez.-


R>  sessionInfo()

R version 3.0.2 Patched (2013-12-11 r64449)
Platform: x86_64-apple-darwin10.8.0 (64-bit)

locale:
[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] graphics  grDevices utils     datasets  parallel  compiler  stats
[8] methods   base

other attached packages:
[1] knitr_1.6.3            ggplot2_1.0.0          slidifyLibraries_0.3.1
[4] slidify_0.3.52

loaded via a namespace (and not attached):
 [1] colorspace_1.2-4 digest_0.6.4     evaluate_0.5.5   formatR_0.10
 [5] grid_3.0.2       gtable_0.1.2     markdown_0.7.1   MASS_7.3-33
 [9] munsell_0.4.2    plyr_1.8.1       proto_0.3-10     Rcpp_0.11.2
[13] reshape2_1.4     scales_0.2.4     stringr_0.6.2    tools_3.0.2
[17] whisker_0.4      yaml_2.1.13

	[[alternative HTML version deleted]]

