[R-sig-ME] Testing for differences in random effects between two groups

[Stephen T]

Hello, 
 
I am trying to apply linear models to some observational data. 
The data are not balanced. 

Basically, I am wish to describe sexual and individual signatures in calls of a bird species. I have many call recordings and measured several acoustic variables from each recording.

Here's an analysis for one variable: 
 
> lmer(MH11~SEX+(1|BIRD)+(1|NIGHT), data=calls) 
Linear mixed model fit by REML  
Formula: MH11 ~ SEX + (1 | BIRD) + (1 | NIGHT)  
   Data: calls  
  AIC  BIC logLik deviance REMLdev 
 3662 3681  -1826     3663    3652 
Random effects: 
 Groups   Name        Variance Std.Dev. 
 NIGHT    (Intercept)  652.77  25.549   
 BIRD     (Intercept) 1083.67  32.919   
 Residual              966.50  31.089   
Number of obs: 356, groups: NIGHT, 138; BIRD, 57 
 
Fixed effects: 
            Estimate Std. Error t value 
(Intercept)  445.059      8.212   54.19 
SEXM          94.779     10.588    8.95 
 
Correlation of Fixed Effects: 
     (Intr) 
SEXM -0.776 
 
SEX is a fixed effect (Female/Male). BIRD (individuals) and NIGHT are random effects. 
The nesting is SEX/BIRD/NIGHT/CALL.

There is a clear difference between-SEXes (fixed effect), i.e. SEXF and SEX
M have different voices.

Secondly, there is some interesting variation between-BIRDs. Individual BIRDs have different voices.


What if the BIRD random effect was not the same in both SEXF and SEXM groups? That would confound the model. Since there is a fixed effect (sexual signature), perhaps the random effect (individual signature) is different between groups SEXF and SEXM as well (unexpected, but should be tested).

I'm not sure how to test for differences between groups in the random effects. One possibility is to split SEXF and SEXM and run separate lmer models. How would I the compare the results? Another is to compare the distributions of random effects between SEXF and SEXM (a two-sample test?). 
 
I would appreciate some advice on how to proceed?

Stephen from Australia.