So a coverage rate now is the probability that the true value of a parameter estimate lies within a confidence interval, like a 95% confidence interval. So it’s basically the number of times you are right or at least pretty close in guessing the value over the total number of times you tried. So take our character Tina again and say she could be in an either a justifiable or unjustifiable dimension. And what is a justifiable dimension you ask? Well, you’ll just to read … you know where I’m going with this, don’t you? Now, say there are 6 justifiable dimensions total out of, lets make it 50 total dimensions. So the probability that Tina would be 6/50 or 12%. And the 95% confidence interval for that probability is (2.99%, 21.01%). I’ll spare you the details but yeah, I did some fancy math stuff to get that. Okay, so actually, I used this site to get that but I’m pretty sure the site used the fancy math stuff. Now, say I generate 100 samples of size 50 and get probabilities of Tina being in a justified dimension between 6% and 26%, all of them associated with their own confidence intervals. Now, doing some fancy math stuff (that I did all my myself — thank you very much — with the help of R), I calculated that and noticed that the confidence intervals of the generated probabilities and the confidence interval of the true probability overlap except for 4.92% of the time, meaning the true probability of Tina being in a justified dimension is covered by the 95% confidence intervals of the probabilities from the generated data 95.08% of the time. Ain’t that cool? And coverage rates can be computed for different parameters obtained from different random number generation methods as well as from different imputation methods. In fact, I used to compute them a lot as part of my dissertation. By the way, a good coverage rate is usually 90% or greater. Funny thing about simulations though is you run this simulation after two, three nights and think you’re done only to find that your coverage rate is 89.99%. Just so you get the gist of my graduate life. But eventually I did start getting those babies at 90% or up after which I did this little dance.
And then came the next challenge: trying to get standardized biases below 50% – which I have decided to cover next time. Until then, turn that frown upside-down, Charlie Brown! Okay, not the most original Peanuts reference … but, um, okay, bye!