How to interpret marginal effect in melogit with odds ratio

[Taka Sakamoto]

Dear Statalisters:

I am trying to correctly interpret the results of marginal effects that are produced with melogit with odds ratio option. Attached below are the commands and results. I tell Stata to calculate marginal effects with the command "margins, dydx(i.singlemom3) at(nordic=(0(1)1)) expression(exp(predict(xb)))". Both singlemom3 and nordic are dummy variables.

In this case, is it correct to interpret the marginal effect of singlemom3 with nordic==1 that when nordic==1, the odds ratio is .0073617, and it suggests that the odds of pov50 happening when nordic==1 are 99.3% lower? I'd like to make sure 2 things: that I'm calculating the odds ratio correctly and that I'm interpreting the meaning of .0073617 correctly.

Thank you for your help in advance!

Best,

Taka

. use pov50 age age2 educ nhhmem17 child5 nordic limatppp_L3 itotedupergdp_L3 lalmptotppp_L3 age singlemom3 cid emp_ilo using $mydata/tsakam/lis15pov2.dta

. melogit pov50 age age2 educ nhhmem17 child5 nordic##singlemom3 limatppp_L3 itotedupergdp_L3 lalmptotppp_L3 if age>24&age<55 & age~=.&emp_ilo==1 || cid: , or vce(cl cid)

Fitting fixed-effects model:

Iteration 0:   log likelihood = -359581.21  
Iteration 1:   log likelihood = -298766.56  
Iteration 2:   log likelihood = -297421.02  
Iteration 3:   log likelihood = -297396.28  
Iteration 4:   log likelihood = -297396.27  

Refining starting values:

Grid node 0:   log likelihood = -296811.38

Fitting full model:

Iteration 0:   log pseudolikelihood = -296811.38  (not concave)
Iteration 1:   log pseudolikelihood = -296805.38  (not concave)
Iteration 2:   log pseudolikelihood = -296799.35  (not concave)
Iteration 3:   log pseudolikelihood = -296794.37  (not concave)
Iteration 4:   log pseudolikelihood = -296790.26  (not concave)
Iteration 5:   log pseudolikelihood = -296786.89  (not concave)
Iteration 6:   log pseudolikelihood =  -296781.2  (not concave)
Iteration 7:   log pseudolikelihood = -296778.86  (not concave)
Iteration 8:   log pseudolikelihood = -296776.93  (not concave)
Iteration 9:   log pseudolikelihood = -296776.13  
Iteration 10:  log pseudolikelihood = -296735.71  
Iteration 11:  log pseudolikelihood =  -296638.1  
Iteration 12:  log pseudolikelihood = -296603.27  
Iteration 13:  log pseudolikelihood =  -296603.2  
Iteration 14:  log pseudolikelihood =  -296603.2  

Mixed-effects logistic regression               Number of obs     =    2017275
Group variable:             cid                 Number of groups  =         14

                                                Obs per group:
                                                              min =      2,091
                                                              avg =  144,091.1
                                                              max =    748,229

Integration method: mvaghermite                 Integration pts.  =          7

                                                Wald chi2(11)     =  987786.79
Log pseudolikelihood =  -296603.2               Prob > chi2       =     0.0000
                                        (Std. Err. adjusted for 14 clusters in cid)
-----------------------------------------------------------------------------------
                  |               Robust
            pov50 | Odds Ratio   Std. Err.      z    P>|z|     [95% Conf. Interval]
------------------+----------------------------------------------------------------
              age |   .8267114   .0442137    -3.56   0.000     .7444413    .9180733
             age2 |   1.002135   .0005695     3.75   0.000     1.001019    1.003251
             educ |   .5662244   .0347937    -9.26   0.000     .5019766    .6386952
         nhhmem17 |   1.159116   .0329424     5.20   0.000     1.096316    1.225514
           child5 |   1.028166    .106204     0.27   0.788     .8397282     1.25889
         1.nordic |   .3562969   .0983028    -3.74   0.000     .2074743    .6118709
     1.singlemom3 |   4.729612   .7387682     9.95   0.000     3.482309    6.423677
                  |
nordic#singlemom3 |
             1 1  |    .315331   .0472083    -7.71   0.000     .2351436    .4228635
                  |
      limatppp_L3 |   1.065768   .0688701     0.99   0.324     .9389829    1.209671
 itotedupergdp_L3 |   1.114404   .0699055     1.73   0.084       .98548    1.260195
   lalmptotppp_L3 |   1.032103    .054843     0.59   0.552     .9300204     1.14539
            _cons |   1.626621   1.818156     0.44   0.663     .1819097    14.54511
------------------+----------------------------------------------------------------
cid               |
        var(_cons)|   .0814162   .0573185                      .0204857    .3235727
-----------------------------------------------------------------------------------
Note: Estimates are transformed only in the first equation.
Note: _cons estimates baseline odds (conditional on zero random effects).

. margins, dydx(i.singlemom3) at(nordic=(0(1)1)) expression(exp(predict(xb)))

Average marginal effects                        Number of obs     =  2,017,275
Model VCE    : Robust

Expression   : exp(predict(xb))
dy/dx w.r.t. : 1.singlemom3

1._at        : nordic          =           0

2._at        : nordic          =           1

-------------------------------------------------------------------------------
              |            Delta-method
              |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
0.singlemom3  |  (base outcome)
--------------+----------------------------------------------------------------
1.singlemom3  |
          _at |
           1  |   .1568204   .0401026     3.91   0.000     .0782208    .2354199
           2  |   .0073617    .001456     5.06   0.000      .004508    .0102154
-------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. 
end of do-file

[Clyde Schechter]

No, the -margins- command you have written is not giving you marginal effects as odds ratios. It is giving you differences in odds, which, I suppose, is legal but is pretty strange. I've never seen anybody report effects as differences in odds. Differences in probabilities are commonly reported, and, for many people, are more comprehensible than odds ratios.

But if you want to report effects in the odds-ratio metric, you don't use -margins- for that. You can do that directly from the -melogit- output. The odds ratio for singlemom3 at nordic = 0 is just what you see in the -melogit- output in the row for 1.singlemom3, namely 4.73 (to 2 decimal places). At nordic = 1, the odds ratio for single mom3 is given by

lincom _b[1.singlemom3] + _b[1.nordic#1.singlemom], or

The result will come out to be approximately 1.35 in the example you show.

[Taka Sakamoto]

Thank you for your explanation and help. 1) Are you suggesting I use differences in probabilities in the regression or just for margins? 2) I'm not familiar with programming, and could you tell me what "L" in "l.singlemom3" means? A lag?

Thank you again for your help.

[Taka Sakamoto]

Oh, I see. It's the number "1", not "L". I'm sorry for my goof.

So, the result of 1.35 means that "odds ratio goes down by "4.73-1.35"? I'm a bit confused.

Thank you for your help.

Taka

[Clyde Schechter]

So, the result of 1.35 means that "odds ratio goes down by "4.73-1.35"?

No.

You have a model with an interaction between singlemom3 and nordic. That means that there is no single odds ratio representing the effect of singlemom3. Rather there are two odds ratios, one that applies when nordic = 0 and the other applying when nordic = 1.

In the logistic regression output, what is shown in the row labeled 1.singlemom3, 4.73, is the odds ratio for singlemom3 that applies when nordic = 0. In the row which is labeled nordic#singlemom3 1 1, is the ratio of the nordic = 1 odds ratio to the nordic = 0 odds ratio. So if you multiply 4.73 by 0.32 you get approximately 1.35. So,1.35 is the odds ratio for singlemom3 that applies when nordic = 1.

Interaction models are confusing at first; it takes a bit of experience with them to reach the point where they are understandable at a glance. Your situation is made even more confusing by virtue of being a logistic model, so that the odds ratios combine by multiplication, rather than the simpler linear regression where the coefficients combine by addition. (It's the same thing, underlyingly, because the odds ratios are the exponentials of the coefficients, and the exponential transforms addition of coefficients to multiplication of odds ratios. But the extra step makes it harder to follow what's going on.)

[Taka Sakamoto]

Thank you for teaching me patiently. So, if I use differences in probabilities, I can use the more conventional interpretation of additive marginal effects, as in simple OLS models? Or it's still different from linear models, because mine is logistic models?

Thank you again for your help.

[Clyde Schechter]

If you use the -margins- command without specifying a -predict()- option, you will get the default prediction, which is probability. Then the -margins- commands will give you predicted probabilities, and for marginal effects (-dydx()- option) you will get differences in probabilities.

[Taka Sakamoto]

Thank you. Would you allow me to ask you if I understand correctly? I'm sorry, I'm new to logit analysis and am feeling desperate and dense.

If my margin command didn't have the predict() option, I could understand the results of margins as single mothers (singlemom3=1) have a 0.007% higher probability of being poor (pov50) when nordic=1 as compared to when nordic=0?

I really appreciate your patient help.

Best,

Taka

[Taka Sakamoto]

I'm sorry. Is It a 0.7% higher probability?

[Clyde Schechter]

If your command had been

 margins, dydx(i.singlemom3) at(nordic=(0(1)1))

then, assuming (counterfactually) that you got the output

-------------------------------------------------------------------------------
              |            Delta-method
              |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
0.singlemom3  |  (base outcome)
--------------+----------------------------------------------------------------
1.singlemom3  |
          _at |
           1  |   .1568204   .0401026     3.91   0.000     .0782208    .2354199
           2  |   .0073617    .001456     5.06   0.000      .004508    .0102154
-------------------------------------------------------------------------------

then the interpretation would be then when nordic == 0 (_at 1) the probability of pov50 wold be 0.157 greater when singlemom3 == 1 than when singlemom3 == 0, and the probability of pov50 when nordic = 1 (_at 2) would be 0.007 greater when singlemom3 == 1 than when singlemom3 == 0.

Added: I prefer to report probabilities in the 0 to 1 range rather than as percents (0 to 100). If, however, you need to report them as percentages, then a 0.157 probability difference corresponds to a difference of 15.7 percentage points (not percent), and a 0.007 probability difference corresponds to a difference of 0.7 percentage points (again, not percent).

[Taka Sakamoto]

Thank you for patiently answering my novice questions. I understand it now. Below are the results of a very close model in log odds. The explanation would be that the probability of pov50 would be 0.128 greater when singlemom3==1 than when singlemom==0 when nordic==0, and that the probability of pov50 when nordic=1 would be 0.01 greater when singlemom3==1 than when it is 0.

Thank you so much for all your help with this. I've learned a lot.

Best wishes,

Taka

. use pov50 age age2 educ nhhmem17 child5 nordic limatppp_L3 itotedupergdp_L3 lalmptotppp_L3 age singlemom3 cid emp emp_ilo using $mydata/tsakam/lis15pov90.dta

. melogit pov50 age age2 educ nhhmem17 child5 nordic##singlemom3 limatppp_L3 itotedupergdp_L3 lalmptotppp_L3 if age>24&age<55 & age~=. &emp_ilo==1 || cid: , vce(cl cid)

Fitting fixed-effects model:

Iteration 0:   log likelihood = -405916.29  
Iteration 1:   log likelihood = -340400.54  
Iteration 2:   log likelihood = -338857.51  
Iteration 3:   log likelihood = -338830.75  
Iteration 4:   log likelihood = -338830.75  

Refining starting values:

Grid node 0:   log likelihood = -338194.39

Fitting full model:

Iteration 0:   log pseudolikelihood = -338194.39  (not concave)
Iteration 1:   log pseudolikelihood = -338180.87  (not concave)
Iteration 2:   log pseudolikelihood = -338167.33  (not concave)
Iteration 3:   log pseudolikelihood = -338156.25  (not concave)
Iteration 4:   log pseudolikelihood =  -338147.2  (not concave)
Iteration 5:   log pseudolikelihood = -338139.78  (not concave)
Iteration 6:   log pseudolikelihood = -338136.74  (not concave)
Iteration 7:   log pseudolikelihood = -338136.14  
Iteration 8:   log pseudolikelihood = -337985.33  
Iteration 9:   log pseudolikelihood = -337835.32  
Iteration 10:  log pseudolikelihood = -337779.63  
Iteration 11:  log pseudolikelihood = -337779.44  
Iteration 12:  log pseudolikelihood = -337779.36  
Iteration 13:  log pseudolikelihood = -337779.31  
Iteration 14:  log pseudolikelihood = -337779.31  

Mixed-effects logistic regression               Number of obs     =    2230029
Group variable:             cid                 Number of groups  =         15

                                                Obs per group:
                                                              min =      2,091
                                                              avg =  148,668.6
                                                              max =    812,650

Integration method: mvaghermite                 Integration pts.  =          7

                                                Wald chi2(11)     =   15696.34
Log pseudolikelihood = -337779.31               Prob > chi2       =     0.0000
                                        (Std. Err. adjusted for 15 clusters in cid)
-----------------------------------------------------------------------------------
                  |               Robust
            pov50 |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
------------------+----------------------------------------------------------------
              age |  -.1881868   .0500605    -3.76   0.000    -.2863036     -.09007
             age2 |   .0020864   .0005171     4.03   0.000     .0010728    .0030999
             educ |  -.5938861    .068825    -8.63   0.000    -.7287806   -.4589915
         nhhmem17 |    .146779    .027108     5.41   0.000     .0936482    .1999098
           child5 |   .0494231   .0933906     0.53   0.597    -.1336192    .2324653
         1.nordic |  -.9954417   .2835791    -3.51   0.000    -1.551246    -.439637
     1.singlemom3 |   1.509881   .1193922    12.65   0.000     1.275877    1.743886
                  |
nordic#singlemom3 |
             1 1  |  -1.028179   .1697999    -6.06   0.000    -1.360981   -.6953774
                  |
      limatppp_L3 |    .120673   .1151112     1.05   0.294    -.1049408    .3462868
 itotedupergdp_L3 |   .0748331   .0841142     0.89   0.374    -.0900277    .2396939
   lalmptotppp_L3 |   .1482272   .1771406     0.84   0.403    -.1989621    .4954165
            _cons |  -.5956198   1.562656    -0.38   0.703    -3.658369     2.46713
------------------+----------------------------------------------------------------
cid               |
        var(_cons)|   .1099331   .0931128                      .0209009    .5782191
-----------------------------------------------------------------------------------

. margins, dydx(i.singlemom3) at(nordic=(0(1)1))

Average marginal effects                        Number of obs     =  2,230,029
Model VCE    : Robust

Expression   : Marginal predicted mean, predict()
dy/dx w.r.t. : 1.singlemom3

1._at        : nordic          =           0

2._at        : nordic          =           1

-------------------------------------------------------------------------------
              |            Delta-method
              |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------+----------------------------------------------------------------
0.singlemom3  |  (base outcome)
--------------+----------------------------------------------------------------
1.singlemom3  |
          _at |
           1  |   .1288889   .0227562     5.66   0.000     .0842876    .1734902
           2  |   .0109097   .0048536     2.25   0.025     .0013968    .0204226
-------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

[Taka Sakamoto]

P.S. Meaning that singlemom3 is associated with lower pov50 when nordic==1 than when nordic==0.

[Taka Sakamoto]

Dear Mr. Schechter:

I was revisiting what you taught me and would like to clarify one thing. The counterfactual example you used in explaining interpretations, is it what it should be when the melogit command is used with "or" option or without "or"?

I'm hoping you would notice this new post.

Thank you for your help.

Clyde Schechter

[Clyde Schechter]

If you use the -or- option with -melogit-, the -melogit- output will show odds ratios rather than coefficients. Odds ratios combine multiplicatively; coefficients (what you get without the -or- option) combine additively. The coefficients are the logarithms of the odds ratios, however, and are not themselves probabilities.

The -margins- command, however, takes no notice of whether your -melogit- command had the -or- option or not. Its default output is predicted probabilities regardless.

[Taka Sakamoto]

Thank you for helping me understand. I just want to make sure I understand correctly. So, whether I do or don't use the -or- option doesn't affect the interpretation of the results of -margins- and of -lincom-? That is, they are probabilities and differences in probabilities, and I don't have to exponentiate the coefficients, correct?

Thank you again for your kind help.

Taka

Clyde Schechter