Binary and Multinomial logistic regression results insignificant

[Alinah Letolo]

Dear all,

I am facing challenges with my model results being insignificant when i am running both binary and multinomial regression models. The p-values for my variables are insignificant. I am using IR(Individual Recode) file from LDHS dataset. my dependent variables are healthcare visits(no/yes), ANC visits(no anc,1 anc, 2 anc, 3 anc, >4anc) and PNC visits(no, yes, don't know). The independent variables are sex of the household(male/female), age(15-24, 25-34,35-44,45-49), marital status(never married, married, widowed, divorced), religion(non Christian, Christian), education(no education, primary, secondary, higher), occupation(non Agric, Agric), wealth index(poor, middle, rich), health insurance(no/yes), distance to health facility(not a problem, a big problem), residence(urban/rural), and anemia level.

I checked multicollinearity between my independent variables and found that education, health insurance and anemia level have VIF greater than 10 and I dropped anemia level from my models but the results are still bad. I also tried grouping my dependent variables into social, economic and other variables and ran adjusted and unadjusted binary logistic models but the more I add more variables the more the p-values become insignificant. could assist me with could be the problem.
below is the output for logistic model with dep. var as visted healthcare facilty=healtcare visits and anc visits as a dependent var. for mlogit


. logistic visited_health_facil i.sex_head i.age i.marital_status i.religion i.occupation i.wealth_index i.education i.health
> _insurance i.anemia_level i.dist_health_facility i.residence, allbaselevels

Logistic regression Number of obs = 1381
LR chi2(21) = 52.56
Prob > chi2 = 0.0002
Log likelihood = -811.04853 Pseudo R2 = 0.0314

--------------------------------------------------------------------------------------
visited_health_facil | Odds Ratio Std. Err. z P>|z| [95% Conf. Interval]
---------------------+----------------------------------------------------------------
sex_head |
1 | 1 (base)
2 | 1.143418 .1782281 0.86 0.390 .8424138 1.551974
|
age |
1 | 1 (base)
2 | 1.387775 .2350928 1.93 0.053 .9956882 1.934261
3 | 1.029242 .1946897 0.15 0.879 .7104041 1.491177
4 | .787294 .1909856 -0.99 0.324 .4893823 1.266559
|
marital_status |
1 | 1 (base)
2 | 2.070133 .3705802 4.06 0.000 1.457547 2.940182
3 | 1.99339 .4860793 2.83 0.005 1.236034 3.2148
4 | 2.13516 .56519 2.87 0.004 1.270903 3.587143
|
religion |
0 | 1 (base)
1 | .7492449 .3411377 -0.63 0.526 .3069477 1.828872
|
occupation |
1 | 1 (base)
2 | 1.042236 .2027846 0.21 0.832 .7117874 1.526095
3 | 1.306453 .3640273 0.96 0.337 .7566884 2.255646
|
wealth_index |
1 | 1 (base)
2 | 1.231303 .2456395 1.04 0.297 .8328255 1.820438
3 | 1.204038 .2239029 1.00 0.318 .8362786 1.733524
|
education |
0 | 1 (base)
1 | 1.822687 1.140474 0.96 0.337 .5346957 6.213234
2 | 2.312652 1.467217 1.32 0.186 .6669326 8.019338
3 | 2.883668 1.903128 1.60 0.109 .7910059 10.51262
|
health_insurance |
0 | 1 (base)
1 | .585914 .2287483 -1.37 0.171 .2725925 1.259371
|
anemia_level |
1 | 1 (base)
2 | .5436657 .4556743 -0.73 0.467 .105171 2.810398
3 | .5967894 .4885422 -0.63 0.528 .1199535 2.969132
4 | .6273136 .5083492 -0.58 0.565 .1281462 3.070885
|
dist_health_facility |
1 | 1 (base)
2 | 1.255869 .1986669 1.44 0.150 .9210692 1.712366
|
residence |
1 | 1 (base)
2 | 1.481787 .2284155 2.55 0.011 1.095403 2.00446
|
_cons | .683371 .7689855 -0.34 0.735 .0753034 6.201529
--------------------------------------------------------------------------------------


and for mlogit

. mlogit anc i.sex_head i.age i.marital_status i.religion i.occupation i.wealth_index i.education i.health_insurance i.dist_h
> ealth_facility i.residence, base(0) allbaselevels

Iteration 0: log likelihood = -834.22496
Iteration 1: log likelihood = -788.39709
Iteration 2: log likelihood = -769.92251
Iteration 3: log likelihood = -766.73995
Iteration 4: log likelihood = -765.43096
Iteration 5: log likelihood = -765.24125
Iteration 6: log likelihood = -765.20904
Iteration 7: log likelihood = -765.20162
Iteration 8: log likelihood = -765.20002
Iteration 9: log likelihood = -765.19969
Iteration 10: log likelihood = -765.19962
Iteration 11: log likelihood = -765.1996

Multinomial logistic regression Number of obs = 1007
LR chi2(72) = 138.05
Prob > chi2 = 0.0000
Log likelihood = -765.1996 Pseudo R2 = 0.0827

--------------------------------------------------------------------------------------
anc | Coef. Std. Err. z P>|z| [95% Conf. Interval]
---------------------+----------------------------------------------------------------
no_anc | (base outcome)
---------------------+----------------------------------------------------------------
1_anc |
sex_head |
1 | 0 (base)
2 | -.329742 .7014485 -0.47 0.638 -1.704556 1.045072
|
age |
1 | 0 (base)
2 | -.4331893 .6905895 -0.63 0.530 -1.78672 .9203413
3 | -.8241518 .9017629 -0.91 0.361 -2.591575 .943271
4 | -22.71515 53965.21 -0.00 1.000 -105792.6 105747.2
|
marital_status |
1 | 0 (base)
2 | -.740034 .7733469 -0.96 0.339 -2.255766 .7756982
3 | -.5553596 1.091913 -0.51 0.611 -2.695469 1.58475
4 | .2287169 1.02628 0.22 0.824 -1.782754 2.240188
|
religion |
0 | 0 (base)
1 | 15.63568 2292.172 0.01 0.995 -4476.939 4508.21
|
occupation |
1 | 0 (base)
2 | -1.080269 1.133853 -0.95 0.341 -3.302579 1.142042
3 | -14.17877 1268.27 -0.01 0.991 -2499.942 2471.584
|
wealth_index |
1 | 0 (base)
2 | .491967 .6932806 0.71 0.478 -.8668381 1.850772
3 | 1.50705 .9012047 1.67 0.094 -.2592792 3.273378
|
education |
0 | 0 (base)
1 | 14.54271 3536.721 0.00 0.997 -6917.303 6946.389
2 | 13.79664 3536.721 0.00 0.997 -6918.049 6945.643
3 | 15.83348 3536.721 0.00 0.996 -6916.013 6947.68
|
health_insurance |
0 | 0 (base)
1 | -.3619751 2590.434 -0.00 1.000 -5077.52 5076.796
|
dist_health_facility |
1 | 0 (base)
2 | -.5606672 .60041 -0.93 0.350 -1.737449 .6161149
|
residence |
1 | 0 (base)
2 | 1.237727 .8231029 1.50 0.133 -.3755252 2.850979
|
_cons | -30.81201 4214.552 -0.01 0.994 -8291.182 8229.558
---------------------+----------------------------------------------------------------
2_anc |
sex_head |
1 | 0 (base)
2 | .1294843 .5480576 0.24 0.813 -.9446889 1.203658
|
age |
1 | 0 (base)
2 | -.5225949 .5352125 -0.98 0.329 -1.571592 .5264022
3 | -.9272794 .6835562 -1.36 0.175 -2.267025 .4124661
4 | -22.70876 40325.85 -0.00 1.000 -79059.92 79014.5
|
marital_status |
1 | 0 (base)
2 | -.2348144 .6023835 -0.39 0.697 -1.415464 .9458355
3 | -.271298 .8733921 -0.31 0.756 -1.983115 1.440519
4 | -.359796 .9233147 -0.39 0.697 -2.16946 1.449868
|
religion |
0 | 0 (base)
1 | 1.27302 1.172576 1.09 0.278 -1.025186 3.571226
|
occupation |
1 | 0 (base)
2 | .9408899 .5775266 1.63 0.103 -.1910415 2.072821
3 | .1172866 1.054219 0.11 0.911 -1.948945 2.183518
|
wealth_index |
1 | 0 (base)
2 | -.3629362 .5734962 -0.63 0.527 -1.486968 .7610957
3 | 1.536379 .6960343 2.21 0.027 .1721765 2.900581
|
education |
0 | 0 (base)
1 | 14.53064 2606.367 0.01 0.996 -5093.854 5122.915
2 | 14.59 2606.367 0.01 0.996 -5093.795 5122.975
3 | 16.1912 2606.367 0.01 0.995 -5092.194 5124.576
|
health_insurance |
0 | 0 (base)
1 | -1.118786 2170.781 -0.00 1.000 -4255.772 4253.535
|
dist_health_facility |
1 | 0 (base)
2 | -.0560826 .4807543 -0.12 0.907 -.9983437 .8861785
|
residence |
1 | 0 (base)
2 | .9513053 .5927119 1.61 0.108 -.2103887 2.112999
|
_cons | -16.42487 2606.367 -0.01 0.995 -5124.81 5091.96
---------------------+----------------------------------------------------------------
3_anc |
sex_head |
1 | 0 (base)
2 | .3657479 .4530789 0.81 0.420 -.5222705 1.253766
|
age |
1 | 0 (base)
2 | -.8335228 .4480069 -1.86 0.063 -1.7116 .0445545
3 | -1.260563 .5548512 -2.27 0.023 -2.348051 -.1730747
4 | -2.803699 1.385748 -2.02 0.043 -5.519715 -.0876828
|
marital_status |
1 | 0 (base)
2 | 1.147784 .5403055 2.12 0.034 .0888047 2.206763
3 | .3858069 .7479268 0.52 0.606 -1.080103 1.851717
4 | .7980124 .7432406 1.07 0.283 -.6587124 2.254737
|
religion |
0 | 0 (base)
1 | 1.244329 .8188686 1.52 0.129 -.3606241 2.849282
|
occupation |
1 | 0 (base)
2 | .4372587 .5049878 0.87 0.387 -.5524992 1.427017
3 | .3387942 .8574201 0.40 0.693 -1.341718 2.019307
|
wealth_index |
1 | 0 (base)
2 | -.0381724 .4409004 -0.09 0.931 -.9023213 .8259764
3 | 1.454559 .5976732 2.43 0.015 .2831414 2.625977
|
education |
0 | 0 (base)
1 | -.1201417 1.530855 -0.08 0.937 -3.120563 2.88028
2 | .1921436 1.545194 0.12 0.901 -2.83638 3.220667
3 | -.9385856 1.984304 -0.47 0.636 -4.82775 2.950579
|
health_insurance |
0 | 0 (base)
1 | -.4433399 1739.733 -0.00 1.000 -3410.256 3409.37
|
dist_health_facility |
1 | 0 (base)
2 | -.1462628 .3913177 -0.37 0.709 -.9132315 .6207058
|
residence |
1 | 0 (base)
2 | .0904916 .4693694 0.19 0.847 -.8294555 1.010439
|
_cons | -.8789839 1.766947 -0.50 0.619 -4.342136 2.584168
---------------------+----------------------------------------------------------------
more_than_4_anc |
sex_head |
1 | 0 (base)
2 | .3921699 .4079782 0.96 0.336 -.4074526 1.191792
|
age |
1 | 0 (base)
2 | -.4984575 .4122573 -1.21 0.227 -1.306467 .3095518
3 | -.5268547 .4895854 -1.08 0.282 -1.486424 .4327151
4 | -2.741161 1.080447 -2.54 0.011 -4.858797 -.6235246
|
marital_status |
1 | 0 (base)
2 | 1.027777 .4681133 2.20 0.028 .110292 1.945262
3 | .0765875 .6221826 0.12 0.902 -1.142868 1.296043
4 | .4058348 .648109 0.63 0.531 -.8644355 1.676105
|
religion |
0 | 0 (base)
1 | 1.171346 .6235753 1.88 0.060 -.0508393 2.393531
|
occupation |
1 | 0 (base)
2 | .2315969 .4573088 0.51 0.613 -.6647119 1.127906
3 | .4891797 .7640979 0.64 0.522 -1.008425 1.986784
|
wealth_index |
1 | 0 (base)
2 | .1614292 .3793904 0.43 0.670 -.5821623 .9050208
3 | 1.896817 .5480776 3.46 0.001 .8226045 2.971029
|
education |
0 | 0 (base)
1 | -.5022687 1.189478 -0.42 0.673 -2.833603 1.829065
2 | -.1430182 1.203612 -0.12 0.905 -2.502055 2.216019
3 | .5765912 1.583689 0.36 0.716 -2.527382 3.680565
|
health_insurance |
0 | 0 (base)
1 | 13.80465 1463.784 0.01 0.992 -2855.159 2882.768
|
dist_health_facility |
1 | 0 (base)
2 | .0789787 .347216 0.23 0.820 -.6015522 .7595096
|
residence |
1 | 0 (base)
2 | .2735681 .4201572 0.65 0.515 -.5499249 1.097061
|
_cons | .5474843 1.358957 0.40 0.687 -2.116023 3.210992
--------------------------------------------------------------------------------------

[Maarten Buis]

Can you show the results from tab visited_health_facil and tab anc and tab visited_health_facil anc, missing ?

[Alinah Letolo]

good day! Maarten and thanks for the reply, here are the results, i included tabulation of pnc too


Tabulation of visited_health_facil

visited health facility last 12m	Freq.	Percent	Cum.
no	2448	37.09	37.09
yes	4153	62.91	100.00
Total	6601	100.00

.
Tabulation of anc

anc visits	Freq.	Percent	Cum.
no anc	120	4.66	4.66
1 anc	41	1.59	6.25
2 anc	132	5.12	11.37
3 anc	357	13.86	25.23
more than 4 anc	1926	74.77	100.00
Total	2576	100.00

Tabulation of pnc

postnatal checks within 2m	Freq.	Percent	Cum.
no	507	19.68	19.68
yes	2057	79.85	99.53
don't know	12	0.47	100.00
Total	2576	100.00

tab visited_health_facil anc, missing

Tabulation of visited_health_facil anc

visited health facility last 12m	anc visits
no anc	1 anc	2 anc	3 anc	more than 4 anc	.	Total
no	50	9	32	94	421	1842	2448
yes	70	32	100	263	1505	2183	4153
Total	120	41	132	357	1926	4025	6601

1926	4025	6601

.

[Maarten Buis]

A model does not create new information, it makes information present in the dataset visible. Unfortunately, your dataset just does not have enough information to estimate that model. When we look at multicollinearity, the variance in the dependent variable (in case of a binary variable, it one outcome has a very small number of observations), the number of observations per explanatory variable, we are looking at different indicators of how much information is present. None of these are in your case awful, but none of these are particularly good, and also small problems add up. You have about 66 observations per variable, which is a number where I would not say that it is impossible to estimate a model, but also a number where you should not be surprised when problems occur. Things like education, wealth, health insurance are correlated, the same with rural/urban residence and distance to health facility. This does apparently not lead to a huge VIF, but it still reduces the amount of information present in the data somewhat. Similarly, a percentage of 37 is not too small per se, but it is small enough to become a problem when combined with other problems.

[Alinah Letolo]

I am not sure if I follow quite well, which other information should the dataset have? what should i do to avoid such problems? the dataset has 6601 observations. I have attached a screenshot of the dataset, if it will somehow help .

[Maarten Buis]

What I was trying to tell you was that I don't think there is anything you can do.

[Janine Stubbs]

Alinah,
I would take a different approach.
When there are (1) a lot of covariates and (2) the functional form of the model is unknown, by which I mean, it is unknown if the model is:
E[Y] = a + bX + cW + dZ + .... etc
or
E[Y] = a + bX*W + cZ + dZ^2 + .... etc
then a respectable workaround in inverse-probability treatment weighting (IPTW), which in your case would actually be inverse-probability exposure weighting, with the exposure being one of the variables sex, marital status, etc.

It only works if the exposure is binary, but the neat thing is that IPTW creates mathematical balance between exposure groups, which means a lot less parameters need to be estimated. For IPTW you need to fit a logistic regression for the probability of the exposure (say you pick marriage status) and then calculate two things: 1. for the exposed: the probability of being married and 2. for the unexposed, the probability of not being married. This leads to generating weights for each individual, which you apply in your regression for the outcome variable: visits to the health facility; this is reduced down to the simplified model: E[Y] = a + bX

For more see: Williamson E.J. (2014) Respirology 19:625-635 and do a lot of googling.

Good luck!
Janine

[Alinah Letolo]

Hi Janine! thank you very much. I will certainly go through your recommendations to gain a deeper understanding of how IPTW can be applied in my analysis. I will get back to you with any questions or insights that may arise while i am applying it.