Test Post: Multi-Level Modelling: LR-tests for 3-level models and saving/exporting random model components

[Johannes Mueller]

Dear all,

I am hoping you can help me address 2 multi-level modeling problems.

1. LR-tests for 3-level models
2. Export of random parts of the model, in particular random-term variance

---

1. LR-tests for 3-level models
I am a degrees of freedom problem with LR-tests for model comparisons when estimating null-models of a three-level model (individuals, company, alliance). The outcome, is dichotomous, level 1 obs: 40000; level 2 obs: 120; level 3 obs: 25. I am aware of the limitations of having only 25 observations at level 3 and do no hypothesize or formally test at that level; a control for nesting. I firstly conduct an overall 3-level LR-test (vs. normal logistic regression), then for each level independently (only company and only alliance random terms, respectively, vs. logistic model) and finally seek to compare the complete 3-level model (both company and alliance RE) to the models containing only either a company or a alliance RE.
The latter part is what causes me headache:

[CODE
meglm DV || Allianceenc: || Companyenc: , family(binomial) link(logit) or iterate(300)
est sto Mod1

meglm DV || Allianceenc: , family(binomial) link(logit) or iterate(300)
est sto Mod2
lrtest Mod2 Mod1

meglm DV || Companyenc: , family(binomial) link(logit) or iterate(300)
est sto Mod3
lrtest Mod3 Mod1
[/CODE]

Upon running this code, I am confused by the error:

  	 		 			Likelihood-ratio test 			LR chi2(1) = 7.21 		 		 			(Assumption: Mod2_FNL4_A nested in Mod1_FNL4_A) 			Prob > chi2 = 0.0072 		 		 			Note: The reported degrees of freedom assumes the 			null hypothesis is not on the boundary 			of 			the 		 		 			parameter space. If this is not true, then 			the reported test is conservative.

---

2. Exporting the random component of the multi-level model

Moreover, I would like to analyze the random part of the model, most importantly the unexplained variance (reduction) for distinct models. Therefore, I would need to save or export the random variance in any way, preferably with a local or scalar such that I can either export them via outreg2 / estout or directly continue the calculation in STATA.

  	 		 			  			  		 		 			Allianceenc 		 		 			var(_cons) .1509835 			.0470435 			.0819808 			.2780655

Would you know how I can save / store / export the random part? It is not part of the estimates that are directly accessible via for example

eret list

---

Thank you very much in advance for taking the time to consider my questions.

Kind regards
Johannes

[Johannes Mueller]

Dear all,

I am hoping you can help me address 2 multi-level modeling problems.

1. LR-tests for 3-level models
2. Export of random parts of the model, in particular random-term variance

---

1. LR-tests for 3-level models
I am a degrees of freedom problem with LR-tests for model comparisons when estimating null-models of a three-level model (individuals, company, alliance). The outcome, is dichotomous, level 1 obs: 40000; level 2 obs: 120; level 3 obs: 25. I am aware of the limitations of having only 25 observations at level 3 and do no hypothesize or formally test at that level; a control for nesting. I firstly conduct an overall 3-level LR-test (vs. normal logistic regression), then for each level independently (only company and only alliance random terms, respectively, vs. logistic model) and finally seek to compare the complete 3-level model (both company and alliance RE) to the models containing only either a company or a alliance RE.
The latter part is what causes me headache:

meglm DV ||  Allianceenc:  || Companyenc: , family(binomial) link(logit)   or   iterate(300)
est sto Mod1
                                
meglm DV ||  Allianceenc:  , family(binomial) link(logit)   or   iterate(300)
est sto Mod2
lrtest  Mod2  Mod1
                                
meglm  DV ||  Companyenc:  , family(binomial) link(logit)   or   iterate(300)
est sto Mod3                                  
lrtest  Mod3  Mod1

Upon running this code, I am confused by the error:

   	 		 			  		 		 			Likelihood-ratio test 			LR chi2(1) = 7.21 		 		 			(Assumption: Mod2_FNL4_A nested in Mod1_FNL4_A) 			Prob > chi2 = 0.0072 		 		 			Note: The reported degrees of freedom assumes the 			null hypothesis is not on the boundary 			of 			the 		 		 			parameter space. If this is not true, then 			the reported test is conservative. 		 		 			invalid syntax 		 		 			r(198);

---

2. Exporting the random component of the multi-level model

Moreover, I would like to analyze the random part of the model, most importantly the unexplained variance (reduction) for distinct models. Therefore, I would need to save or export the random variance in any way, preferably with a local or scalar such that I can either export them via outreg2 / estout or directly continue the calculation in STATA.

  	 		 			  			  		 		 			Allianceenc 			  		 		 			var(_cons) .1509835 			.0470435 			.0819808 			.2780655

Would you know how I can save / store / export the random part? It is not part of the estimates that are directly accessible via for example

eret list

---

Thank you very much in advance for taking the time to consider my questions.

Kind regards
Johannes

[Johannes Mueller]

Multi-Level Modelling: LR-tests for 3-level models and saving/exporting random model components


Dear all,

I am hoping you can help me address 2 multi-level modeling problems.

1. LR-tests for 3-level models
2. Export of random parts of the model, in particular random-term variance

---

1. LR-tests for 3-level models
I am a degrees of freedom problem with LR-tests for model comparisons when estimating null-models of a three-level model (individuals, company, alliance). The outcome, is dichotomous, level 1 obs: 40000; level 2 obs: 120; level 3 obs: 25. I am aware of the limitations of having only 25 observations at level 3 and do no hypothesize or formally test at that level; a control for nesting. I firstly conduct an overall 3-level LR-test (vs. normal logistic regression), then for each level independently (only company and only alliance random terms, respectively, vs. logistic model) and finally seek to compare the complete 3-level model (both company and alliance RE) to the models containing only either a company or a alliance RE.
The latter part is what causes me headache:

meglm DV ||  Allianceenc:  || Companyenc: , family(binomial) link(logit)   or   iterate(300)
est sto Mod1
                                
meglm DV ||  Allianceenc:  , family(binomial) link(logit)   or   iterate(300)
est sto Mod2
lrtest  Mod2  Mod1
                                
meglm  DV ||  Companyenc:  , family(binomial) link(logit)   or   iterate(300)
est sto Mod3                                  
lrtest  Mod3  Mod1

Upon running this code, I am confused by the error:

  	 		 			Likelihood-ratio test 			LR chi2(1) = 7.21 		 		 			(Assumption: Mod2_FNL4_A nested in Mod1_FNL4_A) 			Prob > chi2 = 0.0072 		 		 			Note: The reported degrees of freedom assumes the 			null hypothesis is not on the boundary 			of 			the 		 		 			parameter space. If this is not true, then 			the reported test is conservative.

Since for some reason the HTML posting does not format as I would want it to look (please excuse the format!) here the message again:

Note: The reported degrees of freedom assumes the null hypothesis is not on the boundary of the parameter space. If this is not true, then the reported test is conservative.
invalid syntax r(198);

---

2. Exporting the random component of the multi-level model

Moreover, I would like to analyze the random part of the model, most importantly the unexplained variance (reduction) for distinct models. Therefore, I would need to save or export the random variance in any way, preferably with a local or scalar such that I can either export them via outreg2 / estout or directly continue the calculation in STATA.

  	 		 			  			  		 		 			Allianceenc 			  		 		 			var(_cons) .1509835 			.0470435 			.0819808 			.2780655

(same as above, HTML export from STATA looks not very neat, please find it below again)

Allianceenc
var(_cons) .1509835	.0470435	.0819808	.2780655

Would you know how I can save / store / export the random part? It is not part of the estimates that are directly accessible via for example

eret list

---

Thank you very much in advance for taking the time to consider my questions.

Kind regards
Johannes

[Johannes Mueller]

 Allianceenc
var(_cons) .1509835 .0470435 .0819808 .2780655

[Johannes Mueller]

Likelihood-ratio test
(Assumption: Mod2_FNL4_A nested in Mod1_FNL4_A)               
LR chi2(1) = 7.21
Prob > chi2 = 0.0072
Note: The reported degrees of freedom assumes the null hypothesis is not on the boundary of the parameter space. If this is not true, then the reported test is conservative.
invalid syntax r(198);

[Johannes Mueller]

Dear all,

I am hoping you can help me address 2 multi-level modeling problems.

1. LR-tests for 3-level models
2. Export of random parts of the model, in particular random-term variance

---

1. LR-tests for 3-level models
I am a degrees of freedom problem with LR-tests for model comparisons when estimating null-models of a three-level model (individuals, company, alliance). The outcome, is dichotomous, level 1 obs: 40000; level 2 obs: 120; level 3 obs: 25. I am aware of the limitations of having only 25 observations at level 3 and do no hypothesize or formally test at that level; a control for nesting. I firstly conduct an overall 3-level LR-test (vs. normal logistic regression), then for each level independently (only company and only alliance random terms, respectively, vs. logistic model) and finally seek to compare the complete 3-level model (both company and alliance RE) to the models containing only either a company or a alliance RE.
The latter part is what causes me headache:

meglm DV ||  Allianceenc:  || Companyenc: , family(binomial) link(logit)   or   iterate(300)
est sto Mod1
                                
meglm DV ||  Allianceenc:  , family(binomial) link(logit)   or   iterate(300)
est sto Mod2
lrtest  Mod2  Mod1
                                
meglm  DV ||  Companyenc:  , family(binomial) link(logit)   or   iterate(300)
est sto Mod3                                  
lrtest  Mod3  Mod1

Upon running this code, I am confused by the error:

Likelihood-ratio test
(Assumption: Mod2_FNL4_A nested in Mod1_FNL4_A)               
LR chi2(1) = 7.21
Prob > chi2 = 0.0072
Note: The reported degrees of freedom assumes the null hypothesis is not on the boundary of the parameter space. If this is not true, then the reported test is conservative.
invalid syntax r(198);

---

2. Exporting the random component of the multi-level model

Moreover, I would like to analyze the random part of the model, most importantly the unexplained variance (reduction) for distinct models. Therefore, I would need to save or export the random variance in any way, preferably with a local or scalar such that I can either export them via outreg2 / estout or directly continue the calculation in STATA.

 Allianceenc
var(_cons) .1509835 .0470435 .0819808 .2780655

Would you know how I can save / store / export the random part? It is not part of the estimates that are directly accessible via for example

eret list

---

Thank you very much in advance for taking the time to consider my questions.

Kind regards
Johannes