[R] Cross Validation output

[Donald Catanzaro, PhD]

Good Day All,

I have a negative binomial model that I created using the function 
glm.nb() with the MASS library and I am performing a cross-validation 
using the function cv.glm() from the boot library.  I am really 
interested in determining the performance of this model so I can have 
confidence (or not) when it might be applied elsewhere

If I understand the cv.glm() procedure correctly, the default cost 
function is the average squared error and by running run cv.glm() in a 
loop many times I understand that I can calculate PRESS (PRedictive 
Error Sum of Squares = 1/n*Sum(all PEs) from the default output.

When I run a loop that is 10 times, my PRESS ~25

I have a few questions:

1)  I must now confess my ignorance, how does one interpret my PRESS of 
25 ?  Are there some internet resources that someone could point me to 
to help in the interpretation ?  I've spent most of yesterday studying 
up on things but feel like I am chasing my tail.  Most of the resources 
are either way so heavy in theory that I can't puzzle them out or are a 
couple of paragraphs long and don't have example with data in them.  Is 
my PRESS in essence saying that my model performance is ~ 75% ? (I 
suspect not, but I don't know thus I ask)

2)  All my observations are spatial in nature and thus I would like to 
plot out spatially where the model is performing well and where it is 
not.  This would be somewhat akin to inspecting residuals in OLS. Is 
there a way to output from cv.glm() the PEs for individual data points ? 

3)  My previous idea was to look at AIC, BIC, McFaddenR2 and PseudoR2 as 
Goodness of Fit measures of each subset model.  It appears that I can 
modify the cost function of cv.glm() but I am not to confident in my 
ability to write the correct cost function.  Are there other valid 
measures of GOF for my negative binomial model that I can substitute 
into the cost function of cv.glm() ?  Would anyone care to recommend one 
(or many) ?

Thanks in advance for your patience !

-Don

PS - if you've seen my previous posts, I've abandoned my 80/20 split 
validation scheme.

-- 

-Don 

Don Catanzaro, PhD                  Landscape Ecologist
dgcatanzaro at gmail.com               16144 Sigmond Lane
479-751-3616                        Lowell, AR 72745