Model validation

[Ceciia Villalain]

Hi!
I am externally validating different prognostic models in an independent registry. This database is composed by cases from different centers. I wonder if I need to adjust the different models considering that cases come from 5 different centers. That is, would I just do

logit outcome predictor
or
logit outcome predictor center?

Would this make sense? Is there any other way for adjusting? Should this be done at all?

Thanks!

[Tom Scott]

It would help you to share your data using dataex. I do think that you would want to account for any effect of the center and adjust your standard errors for the fact that observations are nested within centers.

logit outcome predictor i.center, vce(cluster center)

[Ceciia Villalain]

Yes, I guess it makes sense to adjust for center. Thanks!

[Carlo Lazzaro]

Cecilia:
Tom's advice obviously makes sense when the number of clusters is at least 30 (rule of thumb), but with 5 clusters (ie, centers) only the clustered standard errors may be misleading vs. their default counterparts.
See http://cameron.econ.ucdavis.edu/rese...5_February.pdf.

[Tom Scott]

Interesting. Thanks for sharing Carlo Lazzaro. Do you mind giving me the takeaway? Should she just use robust standard errors or is it more complicated?

[Carlo Lazzaro]

Tom:
as many other statistical procedure (say -hausman-), cluster-robust standard errors consistency and efficiency are asymptotical.
Therefore, with few clusters (and with few observations per cluster) the asymtpotics do not kick in.
This issue is well covered in VI. Few Clusters (subparagraphs A, B and C) of the abovementioned source.
A possible fix is represented by different bootstrap approaches.