Change of direction of OR in logistic regression

[Eirin Fraenkl]

Dear stata users,

I am not an expert concerning statistics, so I hope to find some answers in this forum.

I have a survey sample and ran a logistic regression with svy: logistic command. I have an independent variable (depression: yes/no) in the logistic regression, where I'm not sure how to interpret the finding.

The descriptive statistics suggest a positive relation between depression and the outcome variable. This remains true when I run a logistic regression with only depression as independent variable and the outcome variable (OR>1, p<.001).

However, the OR of depression changes direction as soon as I include two items about anxiety (5-point scale) (OR<1, p<.005). I am not sure how to explain this from a statistical point of view or if it is necessary to investigate this change further. I am familiar with independent variables becoming non significant when including other variables in the model, but I have never had a change of direction in the effect.

I checked for multicollinearity before running the regression, there were no VIF>5, however the anxiety items had the highest VIF=4. Could this explain the change in direction? Do I need to exclude the variable from the regression?

I would be happy about any ideas/tips on the matter.

Best

[Carlo Lazzaro]

Eirin:
please share what you typed and what Stata gave you back via CODE delimiters. Thanks.
No wonder about your findings: you ran a simple logististic regression (regressand vs. 1 predictor) and the result differs from a multiple logistic regression (regressand vs. 2 or more predictors).

[Eirin Fraenkl]

Thank you for the response, I will use the CODE delimiters from now on.

That is true, however the results (OR>1) remain the same when I include other predictors as independent variables in the model. The change of direction in OR within the depression variable only occurs with the specific addition of the two anxiety variables.

Regards, Eirin

[Carlo Lazzaro]

Eirin:
it's good that you are intended to use CODE delimters but, as long as you do not, it is really difficult to reply more positively.

[Eirin Fraenkl]

Sorry, I will try to be more clear.

This is the code I used for logistic regression with only one predictor (DREAMING_ge3pw_during is my outcome variable), which generates the following output:

svy: logistic DREAMING_ge3pw_during i.Depression

                                
DREAMING_ge3pw_during | Odds Ratio   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------------+----------------------------------------------------------------
         1.Depression |   1.562158   .0883165     7.89   0.000     1.398296    1.745221
                _cons |   .5071867   .0122425   -28.12   0.000     .4837491    .5317598

As you can see here the Odds Ratio are >1.


And this is the code with anxiety items included as predictors:

svy: logistic DREAMING_ge3pw_during i.Depression i.Anxietya i.Anxeityb

DREAMING_ge3pw_during | Odds Ratio   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------------+----------------------------------------------------------------
         1.Depression |    .915732   .0640418    -1.26   0.208      .798428     1.05027
                      |
               Anxietya |
                   2  |   1.278892   .0948302     3.32   0.001     1.105893    1.478955
                   3  |   1.801229   .1901012     5.58   0.000     1.464629    2.215186
                   4  |   2.013617   .2711141     5.20   0.000     1.546548    2.621744
                   5  |   2.364107   .4404202     4.62   0.000     1.640902    3.406053
                      |
               Anxietyb |
                   2  |   1.082743      .0806     1.07   0.286     .9357444    1.252833
                   3  |    1.24209   .1301081     2.07   0.038     1.011544    1.525182
                   4  |   1.559053   .2078476     3.33   0.001     1.200534    2.024638
                   5  |   1.594419    .286303     2.60   0.009     1.121362    2.267038
                      |
                _cons |   .3731395   .0130837   -28.11   0.000     .3483557    .3996866
---------------------------------------------------------------------------------------

When using this code the switch of Direction in OR occurs.

This switch not occur when I use e.g. the following Code:

svy: logistic DREAMING_ge3pw_during i.Depression i.Finance i.sex
                     

DREAMING_ge3pw_during | Odds Ratio   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------------+----------------------------------------------------------------
         1.Depression |   1.493629   .0862484     6.95   0.000      1.33379    1.672622
                      |
              Finance |
            A little  |   1.013226   .0542017     0.25   0.806     .9123657    1.125236
            Somewhat  |    .898536   .0615238    -1.56   0.118      .785686    1.027595
                Much  |   .8944922   .0713064    -1.40   0.162     .7650977     1.04577
           Very much  |   1.154538   .1246942     1.33   0.183     .9342638    1.426747
                      |
                1.sex |   1.475799   .0724816     7.92   0.000     1.340352    1.624932
                _cons |    .392304   .0192168   -19.10   0.000     .3563892    .4318381

I hope I am using the code correctly and that the description makes more sense now!

Regards

[Rich Goldstein]

while I am not a psychologist, a guess is that anxiety and depression are related to each other and another guess is that "Finance" and gender are not particularly related to depression

[Carlo Lazzaro]

Eirin:
thanks for using CODE delimiters, that male comments on your query easier now:
1) in your first code, -i.Depression- is the only predictor. No wonder that it reaches statistical significance;
2) in your second code, -i.Depression- OR is now<1 (but now you're using more predictors, hence any comparison may make a little sense) but it does not reach statistical significance; hence, you do not have to worry about that;
3) in your third model -i.Depression- OR is >1 (and significantly so) but the other predictors are completely different. Hence, again you cannot compare wisely.

I think that the substantive issue is to include all the predictiors that contribute to give a fair and true view of the data generatiing process you're studying (instead of focusing on -i.Depression- behaviours).

[Eirin Fraenkl]

Thank you for your input.

So if I understand correctly I should not worry about the switch in Odds ratio for one variable - does this also apply if the variable becomes significant?

My model includes a number of variables, so I wanted to break it down a bit to make it more understandable. But my final model actually includes several predictors, and in this model depression actually is significant.

svy : logistic DREAMING_ge3pw_during i.Finance i.sex i.Agegroup_6_ i.Marital i.LivingAlone i.Education i.q12_work_ i.Anxietya i.Anxietyb i.Depression i.Stress i.Nightmares q27_qol q28_qoh WHO_100

 DREAMING_ge3pw_during | Odds Ratio   Std. Err.      t    P>|t|     [95% Conf. Interval]
------------------------+----------------------------------------------------------------
                Finance |
              A little  |   .9639162   .0594898    -0.60   0.552      .854087    1.087869
              Somewhat  |   .8262284     .06569    -2.40   0.016     .7070005    .9655626
                  Much  |   .8201725   .0791348    -2.05   0.040      .678845    .9909228
             Very much  |   .9724825   .1306924    -0.21   0.836     .7472745    1.265562
                        |
                  1.sex |   1.283132   .0734669     4.35   0.000     1.146916    1.435526
                        |
            Agegroup_6_ |
               35-44 y  |   .8660732   .0726412    -1.71   0.086     .7347772     1.02083
               45-54 y  |   .8302914   .0707946    -2.18   0.029     .7025025    .9813259
               55-64 y  |    .745864   .0676661    -3.23   0.001     .6243549    .8910206
                 65+ y  |    .879067   .0946889    -1.20   0.231     .7117508    1.085715
                < 25 y  |   1.051194   .1014297     0.52   0.605     .8700508    1.270051
                        |
                Marital |
        Married/Couple  |    .980936   .0683839    -0.28   0.782     .8556511    1.124565
    Divorced/Separated  |   .9483762   .1159593    -0.43   0.665     .7462693    1.205218
               Widowed  |   .9285018   .1888213    -0.36   0.715     .6232594    1.383237
                        |
            LivingAlone |
                    No  |   .9850024   .1475294    -0.10   0.920     .7344104      1.3211
                        |
              Education |
             Secondary  |   1.164368   .2191419     0.81   0.419     .8051505    1.683851
            Vocational  |   1.089922   .2142434     0.44   0.661     .7414207    1.602235
              Bachelor  |   1.063236   .1964535     0.33   0.740     .7401883    1.527274
                Master  |   1.157658    .221724     0.76   0.445     .7953157    1.685082
              Doctoral  |   1.471892   .2991541     1.90   0.057     .9882374    2.192252
                        |
              q12_work_ |
    At home/ no salary  |   .9107401   .0879502    -0.97   0.333     .7536817    1.100528
        Irregular work  |   .9672281   .0952729    -0.34   0.735     .7974044    1.173219
  Lost job... pandemic  |    1.51084   .3562072     1.75   0.080     .9517397    2.398384
               Retired  |   1.086753   .1182979     0.76   0.445     .8779447    1.345224
      Shift/night work  |   1.006937   .1153855     0.06   0.952     .8043696    1.260518
               Student  |   1.185508   .1153934     1.75   0.080     .9795929    1.434709
       Tempor laid off  |   .8547017   .1794038    -0.75   0.454     .5664122    1.289723
            Unemployed  |    1.27489   .1619402     1.91   0.056        .9939    1.635319
                        |
    
               Anxietya |
                     2  |   1.136345   .0914089     1.59   0.112     .9705853    1.330414
                     3  |    1.49503   .1770546     3.40   0.001     1.185321    1.885662
                     4  |   1.665526   .2502613     3.40   0.001     1.240626    2.235949
                     5  |   1.872981   .4077712     2.88   0.004      1.22237     2.86988
                        |
               Anxietyb |
                     2  |   1.087765   .0883951     1.04   0.301     .9275961    1.275591
                     3  |   1.139355   .1348471     1.10   0.270       .90346    1.436843
                     4  |    1.23307   .1838216     1.41   0.160     .9206291    1.651547
                     5  |   1.208543   .2540827     0.90   0.368     .8003754    1.824865
                        |
           1.Depression |   .7917763   .0692996    -2.67   0.008     .6669548    .9399583
           1.stress_4_5 |   .9974445   .0782285    -0.03   0.974      .855313    1.163195
                        |
          Nightmares |
              frequent  |   4.162579   .4231846    14.03   0.000     3.410513    5.080487
                        |
   
                q27_qol |   1.003059   .0015444     1.98   0.047     1.000037    1.006091
                q28_qoh |   1.003085   .0015393     2.01   0.045     1.000072    1.006107
                WHO_100 |    .994618    .001462    -3.67   0.000     .9917565    .9974878
                  _cons |   .2336148   .0639496    -5.31   0.000      .136608    .3995073
-----------------------------------------------------------------------------------------

Sorry about the confusion.

Kind Regards, Eirin

[Eirin Fraenkl]

Rich:

I still find it weird that the effect should change to the opposite, even if the variables are in some way related (which definetely makes sense). However, there are some other psychological variables included in the model that might relate to depression aswell. So if this explanation makes sense from a statistical point of view, you might be right.

Thanks

[Carlo Lazzaro]

Eirin:
if the flip in the sign of a given coefficients is coupled with statistical significance it's a good habit to investigate what's going on.
That said:
- you do not share how many observations your sample is composed of;
- you may want to test the joint statistical significance of your categorical variables via -testparm-;
- categorizing a continuous predictor (age) is usually not advisable.

[Eirin Fraenkl]

The sample size is n=16.797

Thank your very much for your help, I will have a look into testing the joint statistical significance.

Kind regards,
Eirin

[Carlo Lazzaro]

Eirin:
with such a large sample size so many predictors look ok.