[R] Why is removeSparseTerms() not doing anything?
Mike
mikehall at y7mail.com
Wed Apr 22 20:52:45 CEST 2015
Here's the code and results. The corpus is the text version of a single book. (r vs. 3.2)
> docs <- tm_map(docs, stemDocument)
> dtm <- DocumentTermMatrix(docs)
> freq <- colSums(as.matrix(dtm))
> ord <- order(freq)
> freq[tail(ord)]
one experi will can lucid dream
287 312 363 452 1018 2413
> freq[head(ord)]
abbey abdomin abdu abraham absent abus
1 1 1 1 1 1
> dim(dtm)
[1] 1 5265
> dtms <- removeSparseTerms(dtm, 0.1)
> dim(dtms)
[1] 1 5265
> dtms <- removeSparseTerms(dtm, 0.001)
> dim(dtms)
[1] 1 5265
> dtms <- removeSparseTerms(dtm, 0.9)
> dim(dtms)
[1] 1 5265
>
[[alternative HTML version deleted]]
More information about the R-help
mailing list