Dear list:

I'm hoping to tap in to the statistical expertise in the group,
especially those familiar with simulation techniques. I'm finalizing a
study where I obtain standard errors from two sources. The first source
is a monte carlo simulation and the other source is an analytical model
I have developed that appears to recover the standard errors from the
simulation. All analysis are performed in R using MASS, nlme, and
Matrix.

Here is a very brief description. In the monte carlo, I first sample
from a multivariate distribution to create data. The data are
hypothetical student scores on an achievement test over time and the aim
is to examine what happens to standard errors under certain psychometric
conditions. The data are then "contaminated" to reflect a certain
psychometric problem that occurs in longitudinal analyses of student
achievement scores.

These data are then analyzed using a linear model to obtain parameter
estimates. This is replicated 250 times. 

For example, the model equation used is

Y_{ti} =  \mu + \beta \cdot t + \epsilon_{ti}

So, I obtain 250 estimates of \mu and \beta. I take the standard
deviation of these estimates to get the sampling distribution of the
parameter (standard errors). Next, I take a single data set, contaminate
the scores, and then use the analytical approach to obtain standard
errors. So, I end up with two sets of standards errors, those obtained
under simulated conditions and those obtained from the analytical model.

My question is what are the most acceptable techniques for comparing the
standard errors in order to say that the analytical approach actually
"recovers" the monte carlo standard errors? For the most part, the
standard errors appear to be exactly the same, save rounding error.

One idea I am toying with is to average the standard errors of \mu and
\beta from the simulation and then do a t-test between the two standard
errors which might be something along these lines

t = (SE_{analytical} - SE_{mc} )/  \bar se

Where \bar se is the average of the standard errors.

But I'm not certain this is correct. Can anyone suggest a more
appropriate method for comparing the results?

Many thanks. I can also send a copy of the paper to anyone who would
like more information or details. 

-Harold

	[[alternative HTML version deleted]]


