Calculating the ICC for between class variance after imputation

[Maleika Krüger]

Hi to everyone,

I have a problem with my imputated dataset.
I successfully imputated the values for 32 items (around 10% of the data was missing before) (m=5). Since the data structure contains students nested in Schools I want to calculate the ICC for the data, to see how much variance is located on the school Level.

With the un-imputed data set before I used the loneway command to get the ICC (e.g. loneway item class), but that command is not recognized by the mi estimate prefix for imputed data.

I tried to use the mixed command with the imputed dataset
e.g.:
mi estimate: mixed item || class:
mi estimate: estat icc

but that didn't work either.

Does somebody know of a command to calculate the ICC with imputed data? Or do I have to calculate the ICC for every of the 5 imputation seperately and then combine them? Can I combine them in the first place? Or is the ICC one of the exeptions from Rubin's rules?

Thanks for your help

Minka

[Marcos Almeida]

Hello Minka,

Please check this FAQ:http://www.stata.com/support/faqs/st...and-mi-impute/

There you'll find an example on how to perform the estimation of the ICC.

Hopefully that helps.

Marcos

[Maleika Krüger]

Hi Marcos,

no not really. I already knew the FAQ article. It deals with how to handle clustered data during the imputation process. But I have already imputed the data.

As I said above, I want to calculate the ICC for the items AFTER the imputation process. Maybe I explained it wrong.

Before, since the sutdents are nested in classes, I calculated the ICC to see, how much variance lies on the class Level. I did that with the loneway command. No Problem there.

Since the students had some missing values in the interesting items (item 1-32), I imputed the missing values using multiple imputation in stata.

After I imputed the missing Information, I want to calculate the ICC again for the items 1-32 to see how it changed, now that I have imputated the missing information. To do that I need a command that allows me to run over all imputations (m=5) in all 32 items. Since I cannot use the loneway command anymore, I was wondering if somebody knows another command.

There is one example of how to calculate the ICC after using the mixed command FAQ article. But since I have 32 items and 5 imputations calculating by hand is not very efficient.

Regards,

Minka

[Jeph Herrin]

Here's one trick - get the ICC from each imputed set, and take the average:

mi query
local M=r(M)
scalar icc=0
mi xeq 1/`M' : mixed item || class: ; estat icc ; scalar icc=icc+r(icc2)
scalar icc=icc/`M'
di "ICC:" %5.3f  icc

hth,
Jeph

[Maleika Krüger]

Wow that worked! Thank you so much!

[Steve Samuels]

Please reregister with your full name as the FAQ request. (CONTACT US button at lower right). Thanks

[Weiwen Ng]

I came across this while searching for another topic.

My understanding is that the intra-class correlation is calculated by the variance of the random intercepts divided by total variance. var(_cons), take the square of sd(_cons), gets you the former. The sum of that and the residual variance, i.e. var(Residual), or square of sd(Residual), gets you the latter.

You can invoke the variance option after the mixed command, i.e. mixed y x || cluster:, variance

You can verify the math above by going to an unimputed mixed model, calculating the quantities I said, then comparing that to the result from estat icc.

Now, I know the original question asked about multiply-imputed mixed models. What's confusing me is that the output from mi estimate: mixed ..... || cluster:, variance already gives you variance of the random intercepts and the residual variance, and therefore you can calculate this directly without having to run mixed on each single imputation.

[Lisa Quadt]

Here's one trick - get the ICC from each imputed set, and take the average:

mi query
local M=r(M)
scalar icc=0
mi xeq 1/`M' : mixed item || class: ; estat icc ; scalar icc=icc+r(icc2)
scalar icc=icc/`M'
di "ICC:" %5.3f icc

hth,
Jeph

Thanks a lot for this - I am trying to do what the OP is doing. Would you mind annotating that code please? What is icc2 here? Thanks so much!

[daniel klein]

First time I come across this thread. I share the skepticism that Weiwen implies (at least this is my reading). You could treat the ICC as just another point estimate and simply calculate the average across the completed datasets. Given that the ICC is bound between 0 and 1, I would probably think about some transformation to make its' distribution more symmetrical, but that is not my main point. The ICC is a ratio of variances and, in the MI context, variances are usually made up of the within-dataset and the between-dataset variances. Combining the ICC in the proposed way (i. e., averaging over imputed datasets) might not quite capture both components. If mi estimate provides you with (then correctly combined) estimates for the variances, then I would tend to calculate the ICC from those final results of the mixed model estimates as Weiwen suggests.