That was very helpful. Using the predict.all option I got exactly what I
need. Is there any way of visualizing the predictions? using MDS plot is the
best way?

Also, I want to run random Forest using three different schemes (e.g.
training set 70% and test se 30%, cross validation with k=10, etc). The best
way to do that is using caret package, or has RF any option for that?

Many thanks,

Chrysanthi.





2009/4/28 Liaw, Andy <andy_liaw@merck.com>

>  Let's try an example:
>
> R> iris.1tree <- randomForest(Species ~ ., data=iris, ntree=1)
> R> getTree(iris.1tree, 1)
>   left daughter right daughter split var split point status prediction
> 1             2              3         4        0.80      1          0
> 2             0              0         0        0.00     -1          1
> 3             4              5         4        1.75      1          0
> 4             0              0         0        0.00     -1          2
> 5             6              7         3        4.85      1          0
> 6             8              9         1        6.05      1          0
> 7             0              0         0        0.00     -1          3
> 8             0              0         0        0.00     -1          2
> 9             0              0         0        0.00     -1          3
> R> iris[1,]
>   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
> 1          5.1         3.5          1.4         0.2  setosa
> R> predict(iris.1tree, iris[1,], type="prob")
>   setosa versicolor virginica
> 1      1          0         0
> R> levels(iris$Species)
> [1] "setosa"     "versicolor" "virginica"
> The getTree() function showed the first (and only) tree.  To predict the
> first row of iris, we read the tree in the following way.  In the first row
> (the root node), the variable to split is the 4th, or "Petal.Width".  The
> splitting point is 0.8, so data points with Petal.Width < 0.8 go to left and
> others go to right.  Since the "left daughter" is "2", we look at the second
> row of the tree, and it is a leaf (i.e., a terminal node) since the status
> is -1.  The prediction is "1", or the first level of the factor---
> "setosa".  I don't expect anyone to predict data "manually" like this.
> predict.randomForest() does all this for you.
>
> As to individual tree predictions, predict.randomForest() has an option
> "predict.all" that you can use.  To get the OOB votes, though, you will also
> need to look at the output of randomForest(..., inbag=TRUE) to see which
> data point is OOB for which tree.
>
> I hope that's clear now.
>
> Cheers,
> Andy
>
>
>  ------------------------------
> *From:* Chrysanthi A. [mailto:chrysain@gmail.com]
> *Sent:* Tuesday, April 28, 2009 8:52 AM
>
> *To:* Liaw, Andy
> *Cc:* r-help@r-project.org
> *Subject:* Re: [R] help with random forest package
>
>
> Many thanks for your help. Sorry for my delayed reply, but I was away.
> Regarding the OOB error, sorry it was a typo.
>
> As far as the voting, I was just wondering if there is a function that will
> give me the prediction of each case through each tree. Is there any function
> that produce the rules for each tree? If I have a new case that I want to
> predict the class that it belongs to, how can I predict that? I should look
> to each tree and then get the voting? Or are there some predictive rules
> that I can use? I cannot do that prediction from the results that function
> votes give to me...
>
> Also, I was wondering why randomizations along with combining the
> predictions from the trees significantly improve the overall predictive
> accuracy?
>
> Thanks a lot,
>
> Chrysanthi
>
>
>
>
> 2009/4/13 Liaw, Andy <andy_liaw@merck.com>
>
>>  I really don't understand what you don't understand.  Do you know how a
>> tree forms a prediction?  If not, it may be a good idea to learn about that
>> first.  The code runs prediction of each case through all trees in the
>> forest and that's how the votes are formed.
>>
>> [For OOB predictions, only predictions from trees for which the case is
>> out-of-bag are counted.  That's why you may get odd-ball vote fractions even
>> when you grow 100 trees and expect the votes to be in seq(0, 1, by=0.01).]
>>
>> 100% - 2.34% = 97.66%, not 76.6% (I can only assume you had a typo).
>>
>> Cheers,
>> Andy
>>
>>  ------------------------------
>>  *From:* Chrysanthi A. [mailto:chrysain@gmail.com]
>> *Sent:* Monday, April 13, 2009 9:44 AM
>>
>> *To:* Liaw, Andy
>> *Cc:* r-help@r-project.org
>> *Subject:* Re: [R] help with random forest package
>>
>>
>> But how does it estimate that voting output? How does it get the 85.7% for
>> all the trees?
>>
>> Regarding the prediction accuracy. If I have OOB error = 2.34, then the
>> prediction accuracy will be equal to 76.6%, right?
>>
>> Many thanks,
>>
>> Chrysanthi.
>>
>>
>> 2009/4/13 Liaw, Andy <andy_liaw@merck.com>
>>
>>>  RF forms prediction by voting.  Note that each row in the output sums
>>> to 1.  It says 85.7% of the trees classified the first case as "healthy" and
>>> the other 14.3% of the trees "unhealthy".  The majority (in two-class cases
>>> like this one) wins, so the prediction is "healthy".
>>>
>>> You can take 1 - OOB error rate as the estimate of prediction accuracy
>>> (if you have not selected variables, e.g., using variable importance, in
>>> building the final RF model).
>>>
>>> Andy
>>>
>>>  ------------------------------
>>>  *From:* Chrysanthi A. [mailto:chrysain@gmail.com]
>>> *Sent:* Friday, April 10, 2009 10:44 AM
>>>
>>> *To:* Liaw, Andy
>>> *Cc:* r-help@r-project.org
>>> *Subject:* Re: [R] help with random forest package
>>>
>>>
>>>
>>> Hi,
>>>
>>> To be honest, I cannot really understand what is the meaning of the
>>> votes.. For example having five samples and two classes what the numbers
>>> below means?
>>>       healthy  unhealthy
>>> 1  0.85714286 0.14285714
>>> 2  0.92857143 0.07142857
>>> 3  0.90000000 0.10000000
>>> 4  0.92857143 0.07142857
>>> 5  0.84615385 0.15384615
>>>
>>> Suppose now, having the classification, I have an unknown sample and
>>> according to the results that Ive got, how can I predict in which class it
>>> belongs to? Do the votes give that prediction to us?
>>>
>>> Also,  the error is reported on the "OOB estimate of  error rate", right?
>>> For example, if we have OOB estimate of  error rate:2.34%, we can say that
>>> the prediction accuracy is approx. 97.7%? How can we estimate the prediction
>>> accuracy?
>>>
>>>
>>> Thanks a lot,
>>>
>>> Chrysanthi.
>>>
>>>
>>> 2009/4/8 Liaw, Andy <andy_liaw@merck.com>
>>>
>>>>  I'm not quite sure what you're asking.  RF predicts by classifying the
>>>> new observation using all trees in the forest, and take plural vote.  The
>>>> predict() method for randomForest objects does that for you.  The getTree()
>>>> function shows you what each individual tree is like (not visually, just the
>>>> underlying representation of the tree).
>>>>
>>>> Andy
>>>>
>>>>  ------------------------------
>>>> *From:* Chrysanthi A. [mailto:chrysain@gmail.com]
>>>> *Sent:* Wednesday, April 08, 2009 2:56 PM
>>>> *To:* Liaw, Andy
>>>> *Cc:* r-help@r-project.org
>>>> *Subject:* Re: [R] help with random forest package
>>>>
>>>>   Many thanks for the reply.
>>>>
>>>> So, extracting the votes, how can we clarify the classification result?
>>>> If I want to predict in which class will be included an unknown sample, what
>>>> is the rule that will give me that?
>>>>
>>>> Thanks a lot,
>>>>
>>>> Chrysanthi.
>>>>
>>>>
>>>>
>>>> 2009/4/8 Liaw, Andy <andy_liaw@merck.com>
>>>>
>>>>> The source code of the whole package is available on CRAN.  All
>>>>> packages
>>>>> are submitted to CRAN is source form.
>>>>>
>>>>> There's no "rule" per se that gives the final prediction, as the final
>>>>> prediction is the result of plural vote by all trees in the forest.
>>>>>
>>>>> You may want to look at the varUsed() and getTree() functions.
>>>>>
>>>>> Andy
>>>>>
>>>>> From:  Chrysanthi A.
>>>>>  > Hello,
>>>>> >
>>>>> > I am a phd student in Bioinformatics and I am using the Random Forest
>>>>> > package in order to classify my data, but I have some questions.
>>>>> > Is there a function in order to visualize the trees, so as to
>>>>> > get the rules?
>>>>> > Also, could you please provide me with the code of
>>>>> > "randomForest" function,
>>>>> > as I would like to see how it works. I was wondering if I can get the
>>>>> > classification having the most votes over all the trees in
>>>>> > the forest (the
>>>>> > final rules that will give me the final classification).
>>>>> > Also, is there a
>>>>> > possibility to get a vector with the attributes that are
>>>>> > being selected for
>>>>> > each node during the construction of each tree? I mean, that
>>>>> > I would like to
>>>>> > know the m<<M variables that are selected at each node out of
>>>>> > the M input
>>>>> > attributes.. Are they selected randomly? Is there a
>>>>> > possibility to select
>>>>> > the same variable in subsequent nodes?
>>>>> >
>>>>> > Thanks a lot,
>>>>> >
>>>>> > Chrysanthi.
>>>>> >
>>>>> >       [[alternative HTML version deleted]]
>>>>> >
>>>>> > ______________________________________________
>>>>> > R-help@r-project.org mailing list
>>>>> > https://stat.ethz.ch/mailman/listinfo/r-help
>>>>> > PLEASE do read the posting guide
>>>>> > http://www.R-project.org/posting-guide.html
>>>>> > and provide commented, minimal, self-contained, reproducible code.
>>>>> >
>>>>> Notice:  This e-mail message, together with any attachments, contains
>>>>> information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station,
>>>>> New Jersey, USA 08889), and/or its affiliates (which may be known
>>>>> outside the United States as Merck Frosst, Merck Sharp & Dohme or
>>>>> MSD and in Japan, as Banyu - direct contact information for affiliates
>>>>> is
>>>>> available at http://www.merck.com/contact/contacts.html) that may be
>>>>> confidential, proprietary copyrighted and/or legally privileged. It is
>>>>> intended solely for the use of the individual or entity named on this
>>>>> message. If you are not the intended recipient, and have received this
>>>>> message in error, please notify us immediately by reply e-mail and
>>>>> then delete it from your system.
>>>>>
>>>>>
>>>>  Notice:  This e-mail message, together with any attachments, contains
>>>> information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station,
>>>> New Jersey, USA 08889), and/or its affiliates (which may be known
>>>> outside the United States as Merck Frosst, Merck Sharp & Dohme or
>>>> MSD and in Japan, as Banyu - direct contact information for affiliates is
>>>> available at http://www.merck.com/contact/contacts.html) that may be
>>>> confidential, proprietary copyrighted and/or legally privileged. It is
>>>> intended solely for the use of the individual or entity named on this
>>>> message. If you are not the intended recipient, and have received this
>>>> message in error, please notify us immediately by reply e-mail and
>>>> then delete it from your system.
>>>>
>>>>
>>>
>>>  Notice:  This e-mail message, together with any attachments, contains
>>> information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station,
>>> New Jersey, USA 08889), and/or its affiliates (which may be known
>>> outside the United States as Merck Frosst, Merck Sharp & Dohme or
>>> MSD and in Japan, as Banyu - direct contact information for affiliates is
>>> available at http://www.merck.com/contact/contacts.html) that may be
>>> confidential, proprietary copyrighted and/or legally privileged. It is
>>> intended solely for the use of the individual or entity named on this
>>> message. If you are not the intended recipient, and have received this
>>> message in error, please notify us immediately by reply e-mail and
>>> then delete it from your system.
>>>
>>>
>>  Notice:  This e-mail message, together with any attachments, contains
>> information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station,
>> New Jersey, USA 08889), and/or its affiliates (which may be known
>> outside the United States as Merck Frosst, Merck Sharp & Dohme or
>> MSD and in Japan, as Banyu - direct contact information for affiliates is
>> available at http://www.merck.com/contact/contacts.html) that may be
>> confidential, proprietary copyrighted and/or legally privileged. It is
>> intended solely for the use of the individual or entity named on this
>> message. If you are not the intended recipient, and have received this
>> message in error, please notify us immediately by reply e-mail and
>> then delete it from your system.
>>
>>
> Notice:  This e-mail message, together with any attach...{{dropped:17}}


