[R] ff problems
Bond, Stephen
Stephen.Bond at cibc.com
Tue Apr 3 19:55:47 CEST 2012
this works:
bigcl <- read.table(file="bigCL1.csv",sep=',',header=T , colClasses=c(rep("factor",3),"numeric","NULL","integer",
"numeric","integer","NULL","numeric","NULL"),nrow=1000)
this doesn't:
bigcl <- read.table.ffdf(file="bigCL1.csv",sep=',',header=T ,
colClasses=c(TERMGROUP="factor",SN="factor",
INS="factor",INCENTIVE="numeric",SRC="character",FLAG="integer",
RELAGE="numeric",BURNSUM="integer",prod="character",BAL="numeric",origbal="NULL")
)
Error in ff(initdata = initdata, length = length, levels = levels, ordered = ordered, :
vmode 'character' not implemented
neither this:
> bigcl <- read.table.ffdf(file="bigCL1.csv",sep=',',header=T ,
+ colClasses=c(TERMGROUP="factor",SN="factor",
+ INS="factor",INCENTIVE="numeric",SRC="NULL",FLAG="integer",
+ RELAGE="numeric",BURNSUM="integer",prod="NULL",BAL="numeric",origbal="NULL")
+ )
Error in repnam(colClasses, colnames(x), default = NA) :
the following argument names do not match'SRC','prod','origbal'
if I load as is without classes then:
> bigcl <- read.table.ffdf(file="bigCL1.csv",sep=',',header=T)
> bigcl$term <- factor(bigcl$term)
Error in sort.list(y) : 'x' must be atomic for 'sort.list'
Have you called 'sort' on a list?
attempting to save it:
> ffsave(bigcl,file="fcl")
Error in system(cmd, input = filelist, intern = TRUE) : 'zip' not found
attempting to get rid of the unwanted columns:
> bigcl <- bigcl[,c(-5,-9,-11)]
Error: cannot allocate vector of size 129.8 Mb
In addition: Warning messages:
1: In class(df) <- "data.frame" :
Reached total allocation of 1535Mb: see help(memory.size)
2: In class(df) <- "data.frame" :
Reached total allocation of 1535Mb: see help(memory.size)
3: In class(df) <- "data.frame" :
Reached total allocation of 1535Mb: see help(memory.size)
4: In class(df) <- "data.frame" :
Reached total allocation of 1535Mb: see help(memory.size)
ff was supposed to be gentle on RAM usage??
I am attaching the first 1000 lines from the csv file.
thanks a lot.
Stephen
More information about the R-help
mailing list