[R-sig-ME] Caterpillarplots that compares two models

[Hans Ekbrand]

Dear list,

I have created caterpillarplots that compare the conditional model with 
the full model, see here for an example: 
http://code.cjb.net/country-differences-working.pdf which was based on 
the following two models:

conditional model:
fit.0 <- glmer(poverty.third.year ~ 1 + (1 | country), family = 
binomial("logit"), data = poverty.risks)

full model:
fit.full <- glmer(poverty.third.year ~ 1 + clustering + (1 | country) + 
(1 | country:clustering) + gender + age.in.years + 
age.minus.mean.age.squared + adults.children.ratio + nr.children + 
pl060.others + education.third.year, family = binomial("logit"), data = 
poverty.risks)

In the example above, the random term (1|country) is very much reduced 
in the full model, very few countries differ significantly from the 
mean. These to fits happen to be the result of fitting the models to a 
subsample (those in the population that had been working for at least 6 
months the last year). When the same models are fitted to the complete 
sample, I get a very different view: 
http://code.cjb.net/country-differences-all.pdf

In general, the random effect of county is smaller, but for some 
countries, Estonia, Lithuania and Ireland, the "effect" of country is 
greater in the full model than in the conditional model. That seems 
rather odd to me. What can be the cause of this?

I am a bit worried about the scale of the random effect, perhaps it is 
not the same for the two fits? I guess that would make the whole idea of 
comparing models within one caterpillar plot a rather bad one.


Below I copy the output of (parts of) summary for each fit:

fit.0 (subsample)
Generalized linear mixed model fit by the Laplace approximation
Formula: poverty.third.year ~ 1 + (1 | country)
Data: poverty.risks
AIC BIC logLik deviance
57343 57362 -28670 57339
Random effects:
Groups Name Variance Std.Dev.
country (Intercept) 0.18511 0.43024
Number of obs: 99714, groups: country, 23

Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -2.56422 0.09103 -28.17 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


fit.full (subsample)
Generalized linear mixed model fit by the Laplace approximation
Formula: poverty.third.year ~ 1 + clustering + (1 | country) + (1 | 
country:clustering) + gender + age.in.years + age.squared.transformed + 
adults.children.ratio + nr.children + pl060.others + education.third.year
Data: poverty.risks
AIC BIC logLik deviance
45996 46243 -22972 45944
Random effects:
Groups Name Variance Std.Dev.
country:clustering (Intercept) 0.00901293 0.094936
country (Intercept) 0.00061525 0.024804
Number of obs: 99714, groups: country:clustering, 337; country, 23

Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -2.030e+03 3.758e+02 -5.40 6.61e-08 ***
clusteringEFT-EFT 8.690e-01 6.168e-02 14.09 < 2e-16 ***
clusteringEFT-retired 1.571e+00 5.512e-01 2.85 0.00437 **
clusteringEFT-unemp/inactive 2.363e+00 8.427e-02 28.04 < 2e-16 ***
clusteringEmpl-us 1.342e+00 6.170e-02 21.75 < 2e-16 ***
clusteringEPT 6.855e-01 9.690e-02 7.07 1.50e-12 ***
clusteringInactive-us 2.301e+00 7.672e-02 29.99 < 2e-16 ***
clusteringInto empl 1.227e+00 8.228e-02 14.92 < 2e-16 ***
clusteringRetired 1.528e+00 1.377e-01 11.09 < 2e-16 ***
clusteringSelf 1.862e+00 5.708e-02 32.61 < 2e-16 ***
clusteringSelf-us 1.988e+00 5.864e-02 33.89 < 2e-16 ***
clusteringStud-EFT 1.323e+00 9.211e-02 14.36 < 2e-16 ***
clusteringStud-us 2.473e+00 9.656e-02 25.61 < 2e-16 ***
clusteringUnemp 3.281e+00 7.584e-02 43.27 < 2e-16 ***
clusteringUnemp-us 2.834e+00 5.732e-02 49.43 < 2e-16 ***
gendermale 1.028e-01 2.700e-02 3.81 0.00014 ***
age.in.years 2.922e-02 7.509e-03 3.89 9.96e-05 ***
age.squared.transformed 5.064e+00 9.391e-01 5.39 6.94e-08 ***
adults.children.ratio 1.848e-01 9.994e-03 18.49 < 2e-16 ***
nr.children 3.495e-01 1.675e-02 20.86 < 2e-16 ***
pl060.others -2.118e-02 4.916e-04 -43.08 < 2e-16 ***
education.third.year.L -1.140e+00 3.254e-02 -35.05 < 2e-16 ***
education.third.year.Q -1.496e-01 2.802e-02 -5.34 9.48e-08 ***
education.third.year.C -1.375e-03 2.389e-02 -0.06 0.95410
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

fit.0 (full sample)
Generalized linear mixed model fit by the Laplace approximation
Formula: poverty.third.year ~ 1 + (1 | country)
Data: poverty.risks
AIC BIC logLik deviance
120558 120578 -60277 120554
Random effects:
Groups Name Variance Std.Dev.
country (Intercept) 0.15613 0.39514
Number of obs: 173439, groups: country, 23

Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -2.18302 0.08297 -26.31 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


fit.full (full sample)
Generalized linear mixed model fit by the Laplace approximation
Formula: poverty.third.year ~ 1 + clustering + (1 | country) + (1 | 
country:clustering) + gender + age.in.years + age.minus.mean.age.squared 
+ adults.children.ratio + nr.children + pl060.others + education.third.year
Data: poverty.risks
AIC BIC logLik deviance
101182 101464 -50563 101126
Random effects:
Groups Name Variance Std.Dev.
country:clustering (Intercept) 0.13222 0.36361
country (Intercept) 0.14338 0.37866
Number of obs: 173439, groups: country:clustering, 391; country, 23

Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -3.1631660 0.1268093 -24.94 < 2e-16 ***
clusteringEFT-EFT 0.9268983 0.1253974 7.39 1.45e-13 ***
clusteringEFT-retired 1.0441078 0.1601783 6.52 7.11e-11 ***
clusteringEFT-unemp/inactive 2.2361878 0.1325487 16.87 < 2e-16 ***
clusteringEmpl-us 1.4730953 0.1251873 11.77 < 2e-16 ***
clusteringEPT 0.9096033 0.1553931 5.85 4.81e-09 ***
clusteringInactive 2.2636346 0.1245034 18.18 < 2e-16 ***
clusteringInactive-us 2.1631165 0.1238853 17.46 < 2e-16 ***
clusteringInto empl 1.1975782 0.1413971 8.47 < 2e-16 ***
clusteringRetired 1.5210071 0.1202705 12.65 < 2e-16 ***
clusteringSelf 1.9703456 0.1252232 15.73 < 2e-16 ***
clusteringSelf-us 2.0602333 0.1236974 16.66 < 2e-16 ***
clusteringStud 1.9668401 0.1289621 15.25 < 2e-16 ***
clusteringStud-EFT 1.1877411 0.1450496 8.19 2.64e-16 ***
clusteringStud-us 2.2431522 0.1266391 17.71 < 2e-16 ***
clusteringUnemp 3.4557231 0.1414965 24.42 < 2e-16 ***
clusteringUnemp-us 2.8435813 0.1204452 23.61 < 2e-16 ***
gendermale -0.0333624 0.0177289 -1.88 0.0599 .
age.in.years -0.0100250 0.0009285 -10.80 < 2e-16 ***
age.minus.mean.age.squared 2.7161514 0.3663032 7.42 1.22e-13 ***
adults.children.ratio 0.0845606 0.0074493 11.35 < 2e-16 ***
nr.children 0.3208252 0.0126444 25.37 < 2e-16 ***
pl060.others -0.0228423 0.0003706 -61.63 < 2e-16 ***
education.third.year.L -1.0252936 0.0232188 -44.16 < 2e-16 ***
education.third.year.Q -0.1133282 0.0195783 -5.79 7.10e-09 ***
education.third.year.C 0.0194027 0.0163584 1.19 0.2356
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1