chris.jackson at imperial.ac.uk
Fri Jun 18 15:33:06 CEST 2004
russell alexander wrote:
>I'm writing about msm. It may be that consistent users of Markov models have a good idea as to what constitutes workable data for a model. I think of general rules, in basic statistical studies where n is limited to exclude fairly precise figures in the lower range.
>On the other hand Markov models don't seem to be often enough used for parameters to be as well laid out.
>I also get the feeling that msm is organized to work optimally with certain sizes and shapes of data. Is there a source that anyone is aware of on this? (I have the Nelder text on optimization, and also have a feeling that what's possible is pretty closely connected with optimization questions)
It's very difficult to give general guidelines for how much data is
sufficient to estimate a continuous-time Markov model. There are two
distinct forms of data which these models are used for. The simplest
case is when you have observed the entire trajectory of the process.
In this case, complex transition matrices can sometimes be estimated
with relatively small datasets. However, if you only have
observations at arbitrary times, then certain models will result in very
flat likelihoods. In particular, for models with reversible transitions
(recurrent states) there can be an infinite number of possible paths
followed in between two arbitrary times. Then you will need
substantially more data. Models with non-recurrent states are generally
easier to estimate.
I'd just suggest that you try out the models you are interested in on
your data. Choosing a suitable optimization technique can often help,
but sometimes models are simply over-parameterised. I don't mind
discussing Markov models on r-help, but if you have a question about msm
it's probably safer to mail me directly as the author, as it is an
obscure contributed package which, as far as I am aware, very few people
Christopher Jackson <chris.jackson at imperial.ac.uk>, Research Associate,
Department of Epidemiology and Public Health, Imperial College
School of Medicine, Norfolk Place, London W2 1PG, tel. 020 759 43371
More information about the R-help