[R] Function to "lump" factors together?

[David Winsemius]

On Oct 17, 2011, at 9:45 PM, David Wolfskill wrote:

> Sorry about the odd terminology, but I suspect that my intent might be
> completely missed had I used "aggregate" or "classify" (each of which
> appears to have some rather special meanings in statistical analysis  
> and
> modeling).
>
> I have some data about software builds; one of the characteristics of
> each is the name of the branch.
>
> A colleague has generated some fairly interesting graphs from the  
> data,
> but he's treating each unique branch as if it were a separate factor.
>
> Last I checked, I had 276 unique branches, but these could be
> aggregated, classified, or "lumped" into about 8 - 10 categories; I
> believe it would be useful and helpful for me to be able to do  
> precisely
> that.
>
> A facility that could work for this purpose (that that we use in our
> "continuous build" driver) is the Bourne shell "case" statement.   
> Such a
> construct might look like:
>
> 	case branch in
> 	trunk)    factor="trunk"; continue;;
> 	IB*)      factor="IB"; continue;;
> 	DEV*)     factor="DEV"; continue;;
> 	PVT*)     factor="PVT"; continue;;
> 	RELEASE*) factor="RELEASE"; continue;;
> 	*)        factor="UNK"; continue;;
> 	esac
>
> Which would assign one of 6 values to "factor" depending on the  
> value of
> "branch" -- using "UNK" as a default if nothing else matched.
>
> Mind, the patterns there are "Shell Patterns" ("globs"), not regular
> expressions.
>
> I've looked at R functions match(), pmatch(), charmatch(), and  
> switch();
> while each looks as it it might be coercable to get the result I want,
> it also looks to require iteration over the thousands of entries I  
> have
> -- as well as using the functions in question in a fairly "unnatural"
> way.
>
> I could also write my own function that iterates over the entries,
> generating factors from the branch names -- but I can't help but think
> that what I'm trying to do can't be so uncommon that someone hasn't
> already written a function to do what I'm trying to do.  And I'd  
> really
> rather avoid "re-inventing the wheel," here.

Here's a loopless lumping of random letters with an "other" value .  
There better ways, but my efforts with match and switch came to  
naught. "pmatch" returns a numeric vector that selects the group.

 > x <- sample(letters[1:10], 50, replace =TRUE)
 > c("abc","abc","abc","def","def","def","ghi","ghi","ghi", "j") 
[pmatch(x, letters[1:10], duplicates.ok=TRUE, nomatch=10)]
  [1] "ghi" "ghi" "ghi" "ghi" "ghi" "def" "def" "ghi" "def" "abc"  
"abc" "j"   "def" "def" "ghi"
[16] "abc" "j"   "def" "ghi" "abc" "ghi" "abc" "abc" "abc" "abc" "abc"  
"abc" "ghi" "def" "abc"
[31] "ghi" "def" "ghi" "def" "abc" "ghi" "ghi" "j"   "abc" "def" "abc"  
"ghi" "abc" "def" "def"
[46] "def" "j"   "ghi" "def" "def"

Classifying 5 million letters in about a second:

 > x <- sample(letters[1:10], 5000000, replace =TRUE)
 > system.time( v <- 
c("abc","abc","abc","def","def","def","ghi","ghi","ghi", "j") 
[pmatch(x, letters[1:10], duplicates.ok=TRUE, nomatch=10)] )
    user  system elapsed
   0.858   0.208   1.062

The same strategy (indexing to return a set membership) can be used  
with findInterval.

-- 

David Winsemius, MD
Heritage Laboratories
West Hartford, CT