Measurement Validation

[Daria Erkhova]

Hello!

I have the following question: is it possible to perform measurement validation in Stata? I have about 300 variables in the data (= 60 constructs), some of the items are self-manifested. In order to start the causal analysis I first need to check measurement validation. Before I always used SPSS for these reason by first doing Reliability check (Chronbach's alpha) and EFA; and then CFA in AMOS, where I delete the items with the squared multiple correlations <= 0.4. I do not understand whether it is possible to do the measurement validation in Stata?! Should I follow the same procedure? 1. Reliability check (alpha), 2. EFA (factor) 3. CFA (estat gof??) ? In CFA how do I decide which item should I delete? What do I have to pay attention at, what coefficients?

Thank you!

[wbuchanan]

Daria Erkhova how are you estimating the reliability coefficient without understanding the dimensionality of the instrument? The reliability coefficient for all items will be less useful when your measurement instrument is multidimensional. It is highly doubtful that you truly have 60 dimensions in the single measurement instrument. You may have testlet effects and/or spurious results due to a lack of precision in your parameter estimates; more importantly, a 300 item tool is extremely unlikely to avoid significant missing data issues. That said, you need to provide a more specific definition for validation than what is implied here. For example, predictive validity is different from concurrent and internal/construct validity; so once you have a precise understanding of which form of validity you are interested in it becomes easier to proceed. I'll assume, however, that you're referring to internal/construct validity here.

You develop a bunch of items and pilot your measurement tool. Next, you need to assess the dimensionality of the measurement tool. One thing that is important is to check whether higher order factors are able to model the data as well as many individual factors. If you're able to defend the tool as unidimensional, you can then use IRT to begin pruning/tuning the tool based on the test specifications. If you're working specifically from a CTT perspective, you would use a second sample of observations to validate the EFA using CFA. CFA is not used to determine which items should be deleted, it is confirmatory (e.g., you know what needs to be included in the model a priori). The IRT methods currently available in Stata only support data that conform to a unidimensional model. There are other platforms available that are more specialized to IRT (and multidimensional IRT in particular), but it wouldn't be helpful to suggest those tools without better understanding your specific research goals.