[R] Regarding the 'R' Load Command
Godavarthi, Murali
MGodavarthi at dpscs.state.md.us
Wed May 19 20:43:11 CEST 2010
Hi Steve, Gavin
This is being really helpful. I've pasted the working data, and my test
data below after running the str command on both of those variables. The
working sample actually contains about 300 records, hence I am not able
to paste the whole data here. However my sample test data which I am
trying to get working, is only 1 record, and I've pasted the dput result
below. Datatypes seem to match in both variables for me in terms of
being num/factor. Please suggest where it could be wrong. Thank You!
mytestdata
structure(list(imurder = 0, itheft = 0, irobbery = 0, iassault = 1L,
idrug = 0L, iburglary = 0L, igun = 0L, psych = 0L, Freq = 0L, priors
= 58L, firstage = 19L, intage = 19L, sex = structure(1L, .Label = "1",
class = "factor"), race = structure(1L, .Label = "BLACK", class =
"factor"), marstat = structure(1L, .Label = "SINGLE", class =
"factor"), empac = structure(1L, .Label = "UNEMPLD", class =
"factor"), educ = 0L, zipcode = 21215L, suspendmn = 0L, drugs = 0L,
alco = 0L, probation = 1L, parole = 0L), .Names = c("imurder", "itheft",
"irobbery", "iassault", "idrug", "iburglary", "igun", "psych", "Freq",
"priors", "firstage", "intage", "sex", "race", "marstat", "empac",
"educ", "zipcode", "suspendmn", "drugs", "alco", "probation", "parole"),
class = "data.frame", row.names = "10")
> str(testdata)
'data.frame': 291 obs. of 23 variables:
$ imurder : num 0 0 0 0 0 0 0 0 0 0 ...
$ itheft : num 0 0 0 0 0 1 0 0 0 0 ...
$ irobbery : num 0 0 0 0 0 0 0 0 0 0 ...
$ iassault : num 1 0 1 0 0 0 0 0 0 0 ...
$ idrug : num 0 1 0 1 1 0 0 1 1 1 ...
$ iburglary: num 0 0 0 0 0 0 0 0 0 0 ...
$ igun : num 0 0 0 0 0 0 0 0 0 0 ...
$ psych : num 0 0 0 0 0 0 0 0 0 0 ...
$ Freq : num 0 0 0 0 0 0 0 0 0 0 ...
$ priors : num 58 4 2 0 6 22 0 36 0 0 ...
$ firstage : num 19 39 28 0 49 32 0 24 0 55 ...
$ intage : num 19 39 28 25 49 32 32 24 30 55 ...
$ sex : Factor w/ 2 levels "1","2": 1 2 1 2 2 1 1 1 1 1 ...
$ race : Factor w/ 5 levels "WHITE","BLACK",..: 2 2 1 1 2 1 1 2 2 2
...
$ marstat : Factor w/ 7 levels "SINGLE","MARRIED",..: 1 2 2 1 2 4 7 1
7 3 ...
$ empac : Factor w/ 6 levels "EMPLD FT","EMPLD PT",..: 3 4 3 3 3 3 6
3 6 3 ...
$ educ : num 0 0 0 1 0 0 0 0 0 1 ...
$ zipcode : num 21215 21217 21223 21223 21217 ...
$ suspendmn: num 0 600 0 0 60 3 2 479 0 3 ...
$ drugs : num 0 1 0 0 0 1 0 0 0 1 ...
$ alco : num 0 0 0 0 0 1 0 0 0 1 ...
$ probation: num 1 1 0 0 1 1 1 1 0 1 ...
$ parole : num 0 0 0 0 0 0 0 0 0 0 ...
>
>
>
>
> str(mytestdata)
'data.frame': 1 obs. of 23 variables:
$ imurder : num 0
$ itheft : num 0
$ irobbery : num 0
$ iassault : num 1
$ idrug : num 0
$ iburglary: num 0
$ igun : num 0
$ psych : num 0
$ Freq : num 0
$ priors : num 58
$ firstage : num 19
$ intage : num 19
$ sex : Factor w/ 1 level "1": 1
$ race : Factor w/ 1 level "BLACK": 1
$ marstat : Factor w/ 1 level "SINGLE": 1
$ empac : Factor w/ 1 level "UNEMPLD": 1
$ educ : num 0
$ zipcode : num 21215
$ suspendmn: num 0
$ drugs : num 0
$ alco : num 0
$ probation: num 1
$ parole : num 0
>
Best Regards,
Murali Godavarthi
410-585-3746 (w)
ITCD - DPSCS Data Mining
-----Original Message-----
From: Gavin Simpson [mailto:gavin.simpson at ucl.ac.uk]
Sent: Wednesday, May 19, 2010 12:58 PM
To: Steve Lianoglou
Cc: Godavarthi, Murali; r-help at r-project.org
Subject: Re: [R] Regarding the 'R' Load Command
I think the answer is clear from the error: R thinks the type of data in
the components of 'testmurali' do not match those of the data used to
fit the original randomForest.
The OP should go back to his model fitting code and do
str(obj)
where 'obj' is the name of his original data object used to fit the
randomForest and compare it with
str(testmurali)
to see why the types of data are different. Look for variables that were
factors or characters in one data set and numeric/integer in the other.
This smells like a data import issue...
If revelation still doesn't occur Murali, *please* follow Steve's
suggestions and post and message that shows exactly (i.e. the R code
executed) along side a data set *we* can load into R without jumping
through hoops or having to divine what your data look like using a
crystal ball or ESP.
HTH
G
On Wed, 2010-05-19 at 12:24 -0400, Steve Lianoglou wrote:
> Hi Murali,
>
> I'm sorry, but you're making this too difficult to provide any help.
> Describing what your data structures are and contain is too tedious to
> follow, and end up being rather ambiguous anyway.
>
> My first guess: by your error message, perhaps the columns of the data
> to predict on are different than the training.
>
> If you want to get help, please provide your data in a form that we
> can test against. Look at this post by Hadley Wickham to help you do
> that:
> http://gist.github.com/270442
>
> In particular, not the use of "dput" that you should use so we can
> paste the result in our workspace and get the data objects you try to
> describe. So, to be clear: send us a chunk of text we can paste into R
> that will recreate a workspace that can reproduce your problem. Please
> trim your data files to be only as large as necessary (eg. provide 3
> observations instead of 300)
>
> Thanks,
> -steve
>
> On Wed, May 19, 2010 at 11:40 AM, Godavarthi, Murali
> <MGodavarthi at dpscs.state.md.us> wrote:
> > Hi Steve,
> >
> > Thanks so much for your inputs! I was actually trying to implement
your
> > suggestions, I get the below error (please see the results of
predict
> > command below).
> >
> > What we are trying to do is to feed in values for about 23
> > characteristics of an individual, and use the randomForest()
function to
> > determine if the individual is a violent offender. Expected output
is 0
> > or 1, indicating yes/no.
> >
> > Am I going wrong again? Here is what I was doing:
> >
> > 1) Created a text file with following data:
> > "imurder" "itheft" "irobbery" "iassault" "idrug" "iburglary" "igun"
> > "psych" "Freq" "priors" "firstage" "intage" "sex" "race" "marstat"
> > "empac" "educ" "zipcode" "suspendmn" "drugs" "alco" "probation"
"parole"
> > "10" 0 0 0 1 0 0 0 0 0 58 19 19 "1" "BLACK" "SINGLE" "UNEMPLD" 0
21215 0
> > 0 0 1 0
> >
> > The above format in which the text file was created is in the same
> > format as the one which is already working, but has characteristics
of
> > about 290 individuals fed-in instead of just one individual as
above.
> > Not sure why this doesn't work!
> >
> >
> > 2) Executed the below command sequence:
> >
> >
> >> library(randomForest)
> > randomForest 4.5-34
> > Type rfNews() to see new features/changes/bug fixes.
> >
> >> load("C://Program Files//R//R-2.10.1//bin//rfoutput")
> >
> >> testmurali<-read.table("ex.data",T)
> >
> >> load(testmurali)
> > Error in load(testmurali) : bad 'file' argument
> >
> >> load("testmurali")
> >
> >> names(testmurali)
> > [1] "imurder" "itheft" "irobbery" "iassault" "idrug"
> > "iburglary" "igun" "psych" "Freq" "priors"
> > [11] "firstage" "intage" "sex" "race" "marstat"
"empac"
> > "educ" "zipcode" "suspendmn" "drugs"
> > [21] "alco" "probation" "parole"
> >
> >> predict(rfoutput,newdata=testmurali,type="response")
> > Error in predict.randomForest(rfoutput, newdata = testmurali, type =
> > "response") :
> > Type of predictors in new data do not match that of the training
data.
> >>
> >
> > The model rfoutput used in the above predict command is also based
on a
> > working example with similar data.
> >
> >
> > Also, does load command accept a data string input directly (without
> > storing it into a file and then providing path of the file as a
string)?
> >
> > Please suggest. Thanks in advance!
> >
> >
> > Best Regards,
> > Murali Godavarthi
> > 410-585-3746 (w)
> >
> > ITCD - DPSCS Data Mining
> >
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: Steve Lianoglou [mailto:mailinglist.honeypot at gmail.com]
> > Sent: Tuesday, May 18, 2010 4:13 PM
> > To: Godavarthi, Murali
> > Cc: r-help at r-project.org
> > Subject: Re: [R] Regarding the 'R' Load Command
> >
> > Hi,
> >
> > On Tue, May 18, 2010 at 2:49 PM, Godavarthi, Murali
> > <MGodavarthi at dpscs.state.md.us> wrote:
> >> Hi,
> >>
> >> I'm new to 'R' and need some help on the "Load" command. Any
responses
> >> will be highly appreciated. Thanks in advance!
> >>
> >> As per manuals, the "Load" command expects a binary file input that
is
> >> saved using a "save" command.
> >
> > Or a path to the file ...
> >
> >> However it is required that we need to
> >> call the 'R' program from
> >>
> >> Java web application using RJava, and pass a string to the 'R"
program
> >> instead of a binary file. Is it possible?
> >
> > Yes, pay closer attention to the description for the "file" argument
> > in the load function (see ?load):
> >
> > """a (readable binary) connection **or a character string** giving
the
> > name of the file to load"""
> >
> > (emphasis mine)
> >
> >> I was exploring the options of using TextConnections, file
connections
> >> and other types of connections in order to read a stream of input
> >> (either from a file, stdin etc). I am able to read the string, but
the
> >> Save and Load commands are not accepting the string input. Here is
the
> >> sequence of commands I tried running, and the error received. There
is
> >> no clue on this error, especially when trying to use the eval
function
> >> in randomForest package, even on the internet. Can anyone help
please!
> >>
> >>> library(randomForest)
> >>
> >> randomForest 4.5-34
> >>
> >> Type rfNews() to see new features/changes/bug fixes.
> >>
> >>
> >>
> >>> load("C://Program Files//R//R-2.10.1//bin//rfoutput")
> >>
> >>
> >>
> >>> zz <- file("ex.data", "w")
> >>
> >>
> >>
> >>> cat("\"imurder\" \"itheft\" \"irobbery\" \"iassault\" \"idrug\"
> >> \"iburglary\" \"igun\" \"psych\" \"Freq\" \"priors\" \"firstage\"
> >> \"intage\" \"sex\" \"race\" \"marstat\" \"empac\"
> >>
> >> \"educ\" \"zipcode\" \"suspendmn\" \"drugs\" \"alco\" \"probation\"
> >> \"parole\"",file = zz, sep = "\n", fill = TRUE)
> >>
> >>
> >>
> >>> cat("\"10\" 0 0 0 1 0 0 0 0 0 58 19 19 \"1\" \"BLACK\" \"SINGLE\"
> >> \"UNEMPLD\" 0 21215 0 0 0 1 0",file = zz, sep = "\n", fill = TRUE)
> >
> > What are you trying to do here?
> > It looks like you want to save a table of sorts. First create your
> > data into a data.frame, then save that data.frame to a file using
> > write.table (or write.csv, etc).
> >
> >>> save(zz, file = "testmurali", version = 2)
> >
> > You're saving a file "object" here, not the contents of the file.
> > Once you successfully serialize your data into a text file, just
load
> > it from "like normal" using read.table (or similar).
> >
> > Anyway, I'm not sure what we're talking about here, but in short:
> >
> > 1. You need to make sure that you are correctly saving what you
think
> > you're saving.
> > 2. You can pass a character string to the `load` function, so you
can
> > send it through (over from) java as you wich.
> > 3. I don't think you really want to deal with load/save here,
because
> > it looks like you are dealing with some tab delimited file -- in
which
> > case use read.table (or similar) and load it that way. You can, of
> > course, still use save/load, but make sure you save/load the right
> > thing (not a file object like you're doing here).
> >
> > --
> > Steve Lianoglou
> > Graduate Student: Computational Systems Biology
> > | Memorial Sloan-Kettering Cancer Center
> > | Weill Medical College of Cornell University
> > Contact Info: http://cbio.mskcc.org/~lianos/contact
> >
>
>
>
--
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%
Dr. Gavin Simpson [t] +44 (0)20 7679 0522
ECRC, UCL Geography, [f] +44 (0)20 7679 0565
Pearson Building, [e] gavin.simpsonATNOSPAMucl.ac.uk
Gower Street, London [w] http://www.ucl.ac.uk/~ucfagls/
UK. WC1E 6BT. [w] http://www.freshwaters.org.uk
%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%~%
More information about the R-help
mailing list