Interpreting marging plots and interactions

[Alejandro Torres]

Dear statalisters,

I would like to ask you please some help interpreting interactions and marginplots.

What I did is

fracreg logit participation RightLeftdummy##c.values

participation is the % of people participating on environmental issues
rightleft dummy is a dummy=1 if people are left ideology oriented and 0=right.
Values is % values on a society

the coefficient are:

 RightLeftdummy: coefficient 11.076 std error 0.9364 95% Conf. Interval 9.2402 to 12.9110

 Values:coefficient 9.660 std error 0.7925 95% Conf. Interval 8.1067 to 11.2132

  RightLeftdummy#c.values: coefficient -12.8800 std error 1.0959 95% Conf. Interval -15.028 to -10.7321

Then I use:

 
margins RightLeftdummy, at(values =(0.442(0.05)0.9487)) 

marginsplot, xdimension(at(values))

What I got is:

My questions then are:

How should I interpret the interaction coefficient? I understand that says "The effect of values change on people participation identified with left ideology is on average 12.8 points lower, than the corresponding effect of values on people participation identified with right ideology. Is that corrrect?

My second question is about how should I interpret the plot? why is the people from left ideology decreasing in the plot? How can I see that trend in the regression only?

Finally, If I add a second interaction, the interpretation of the coefficient of both interactions are the same?

I really really appreciate your help on this, since I have someone telling me that the interaction says that the values are moderating negatively the people participating and values seem to work as a substitute for personal political ideology, but I don't know if that is correct, how can I know if values are substitutes or complements? I am not sure that I they told me is correct.

Thank you very much again.

Alejandro.

[Clyde Schechter]

How should I interpret the interaction coefficient? I understand that says "The effect of values change on people participation identified with left ideology is on average 12.8 points lower,than the corresponding effect of values on people participation identified with right ideology. Is that corrrect?

No, that's not correct. The coefficients are not in the probability metric, they are in the log odds ratio metric. Moreover, for the interaction term, the coefficient represents the difference in log odds. It is very difficult to give an interpretation in simple, understandable terms. You are better off pretty much ignoring the -fracreg- output and focusing on the -marginsplot- graph, which really does all the hard work for you, including putting the results into the more easily understood probability metric.

My second question is about how should I interpret the plot? why is the people from left ideology decreasing in the plot? How can I see that trend in the regression only?

That -marginsplot- output shows you that there is an extremely strong interaction between right left and values in society here: the two curves go in opposite directions and, except where they intersect, the they separate pretty widely. For those with right ideology, participation is low when values are low and rises as values increase. For those with left ideology the opposite is true: when values are low participation is high, and with increasing values we see decreasing participation.

As for how you see that in the regression, let's parse out what the regression equation is and what it means for both ideologies. The basic regression equation you are fitting is:

log odds participation = constant + 11.076*rightleft + 9.660*values - 12.88*rightleft*values

The variable rightleft has the value 1 for left ideology and 0 for right ideology. So for those with right ideology, both the rightleft and rightleft*values terms are zero. Consequently, the relationship between participation and values for those with right ideology is:

log odds participation = constant + 9.660*values

This is consistent with what the graph shows: as values increases, so does participation.

What about those with left ideology. In this case the value of rightleft is 1, so the equation for them is:

log odds participation = constant + 11.076 + 9.660*values - 12.88*1*values

which simplifies to

log odds participation = constant - 3.22*values

Again, this supports what you see in the graph. For those with left ideology, the net effect of values is negative: participation declines as values increases. So this phenomenon arises because the interaction coefficient is a larger negative than the positive coefficient of values.That's where you see it in the regression output.

Finally, If I add a second interaction, the interpretation of the coefficient of both interactions are the same?

Maybe, maybe not. If the second interaction involves any of the variables in the first interaction, then it is a bit more complicated than that.

I have someone telling me that the interaction says that the values are moderating negatively the people participating and values seem to work as a substitute for personal political ideology, but I don't know if that is correct, how can I know if values are substitutes or complements? I am not sure that I they told me is correct.

Your model provides a description of the relationship between values in society and participation, a separate such relationship for those on the right and those on the left ideologically. Talking about things like "substitutes" or "complements" is moving into a different framework: mechanism and causality. A descriptive model, by itself, does not address these issues. They are outside the scope of a descriptive model.

[Alejandro Torres]

Dear Clyde,

Once again, I just can't express my gratitude for your clear answer.
Please, let me just ask you one final question.


The two interaction analysis is:
Dependent variable is the same, just added the variable demonstration, that is the % of people in a country participating in peaceful demonstrations.

 RightLeftdummy:  coefficient 10.203  Conf. Interval  8.438 to 11.976

 Values:  coefficient 8.011  Conf. Interval  6.379 to 9.643

 Demonstration:  coefficient 0.789  Conf. Interval  0.574 to 1.003

 c.Values##RightLeftdummy:  coefficient -12.929  Conf. Interval  -15.037 to -10.821

 c.Demonstration##RightLeftdummy:  coefficient 0.935  Conf. Interval  0.627 to 1.243

Then, I am sorry for asking you again, how is the interpretation of both interaction? considering that the variable of the personal ideology is repeated in both interactions?

I promise that is my last question.
Thank you so much again, I really appreciate it.

[Clyde Schechter]

So, this is a complicated model to interpret, and to make any sense of it we definitely need to use -margins- to take us to the probability metric. To do that, you will need to select "interesting" values of the Values and Demonstration variables. By "interesting," I mean values that more or less span the practically important range of those variables. For purposes of illustration here, I will assume that Values ranges from about 0.4 to about 0.9 (based on your graph), and I will assume that Demonstration ranges from about 0 to 10. Then you can do two kinds of plots:

margins rightleftdummy, at(Values = (0.4(0.1)0.9) Demonstration = (0(2)10))
marginsplot, xdimension(Values)
marginsplot, xdimension(Demonstration)

(Note: if you are running an older version of Stata you may need to re-run -margins- before you can do the second -marginsplot-.)

The first -marginsplot- will give you separate pairs of curves for different values of Demonstration (each pair of curves includes one for right and the other for left ideology). If this were a linear regression, the curves for different values of Demonstration would not be very interesting: they would look the same, just displaced vertically from each other. But because the logistic model is non-linear, these curves will actually have somewhat different shapes as well. The second -marginsplot- will give you pairs (one right, one left) of curves corresponding to different values of Values. Again, because of the non-linearity of the logistic function, these curves will not only be displaced from each other, but the shapes will be somewhat different.

I think this kind of graphical view is the best way to understand models like this.

[Alejandro Torres]

Dear Clyde,

thank you so much again, I just did the plots and are clear about that you said.
I don't have any further question, just saying again thank you for taking the time helping me and helping us.

Best regards !!
Alejandro

[Alejandro Torres]

Dear Clyde,

I am writing you because I was asked from one of member of the dissertation committee to cite a paper that do the same that I am trying to do here, he said is going to help to support my analysis, but, I can't find any paper yet, do you know any please? At the same time, other member asked me this " your m

odel does not include a 3-way interaction but two 2-ways interactions: ideology x value & ideology x demonstration. I am wondering whether your graph is capturing the 2-ways interaction or a 3-way interaction." Here, to be honest, I am not clear about what is asking me.
Thank you very much for any help.

Best regards,

Alejandro

[Clyde Schechter]

I was asked from one of member of the dissertation committee to cite a paper that do the same that I am trying to do here, he said is going to help to support my analysis, but, I can't find any paper yet, do you know any please?

I'm not sure what you mean by "the same that I am trying to do here." You are doing many things here, and I don't know what specifically this person is referring to. In any case, there is nothing in this thread that can't be found in a basic textbook on regression analysis.

At the same time, other member asked me this " your m
odel does not include a 3-way interaction but two 2-ways interactions: ideology x value & ideology x demonstration. I am wondering whether your graph is capturing the 2-ways interaction or a 3-way interaction." Here, to be honest, I am not clear about what is asking me.

Your model is, indeed, two overlapping twoway interactions. So, if you have three variables, A, B, and C, and you have a model with A##B and A##C, you have two interactions that overlap on variable A. And -marginsplot- just plots the results of those analyses you have done: it doesn't add anything new. But another way to model is with a three way interaction: A##B##C. This is a different model.

When you have two two-way interactions you are saying that the effects of A depend on the values of variables B and C, and the effects of B and C depend on the value of A. But the effects of B and C do not depend on each others' values.

In a three way interaction, each of the variables (A, B, and C) effects depends not just on one of the others alone, but on the combination. In other words, for example, the effect of A not only depends on B and depends on C, but the way in which B modifies the effect of A also depends on C. (And you can switch the roles of A, B, and C in any possible way and that statement will still be true.)

So the use of a three way interaction brings a lot of additional effect modification into the model. Three way interaction models are difficult to understand and people tend to use them only when there is clear theoretical indication that all of that complexity is really there and needed to accurately model the data generating process. So to decide whether A##B A##C is sufficient or you need to go to A##B##C, you will need to carefully consider what the relationships among these variables and the outcome variable are in the real world. It's not a statistical question; it's about understanding the world you are modeling.

[Alejandro Torres]

Thank you so much Clyde, the explanation about the two and three way interaction is absolutely clear.

Now, about the same paper was referred that used the same model as mine, I mean

A##B and A##C. Do you have something in mind? and about a three way do you know any paper too to see the differences in the plots?

Finally, about "Your model is, indeed, two overlapping twoway interactions. So, if you have three variables, A, B, and C, and you have a model with A##B and A##C, you have two interactions that overlap on variable A." that does not mean that this is the same that the two interactions

A##B and A##C simply overlaped right? Because I have a plot as you described before "

The first -marginsplot- will give you separate pairs of curves for different values of Demonstration (each pair of curves includes one for right and the other for left ideology)." I could add to your description that I have separate pairs of curves for different values of demonstration for each ideology and in the X axis the range of values for "Values".

Thank you so much again for taking the time.

Alejandro

[Alejandro Torres]

Dear Clyde, this is one of the plots, what do you think please?

[Clyde Schechter]

It looks to my eye like there really isn't much interaction here. All of the lines show a very gentle upward slope, except for one, the light grey one (which I think is valueave4_0a1 = 0.842, Dpi2017RLcon2. Even that one has only a very slightly negative slope. Unless there is some real practical importance to that one deviation, I would be inclined to disregard the interactions here and re-estimate the model without this interaction. Caveat: I don't know what these variables are, and if I did, it is probably outside my domain of expertise. So if there is some important theoretical reason to include these interactions, then, by all means, keep them in the model. Or if your purpose here is to see if some previous study replicates and they used interactions in their analysis, then you must do that as well. But if this is, as I am assuming, a new attempt at modeling this data with little or no guidance by precedent or theory, I would say that the interaction exhibited by this data is probably negligible.

[Alejandro Torres]

Dear Clyde,

I am sorry I didn't answer before, I was trying to convince to my advisors that this was not a 3 way plot but I couldn't so I was asked to do a plot from the regression but separating in two plots the effects of ideologies and values and in other plot ideologies and demonstration, they said its going to be more clear. I am not convinced, because it is stil a two two way analysis, but whay can I do.....

Thank you very much, again I just can't express my gratitude.

Best regards,

Alejandro

[Alejandro Torres]

Dear Clyde,

Expresing once again my gratitude, I am writing you again, I think I am in the final part of my questions and analysis, I am just trying to convince my advisor that my analysis is correct in terms of methods now, theoretically speaking it looks interesting at least.

I am back to the beginning of this post to ask you, when I see the signifcance of the interaction, what does it mean? (I understand that its better to see the plots for analysis, and that is what I am doing) but, let me try to explain, if I have a p-value (0.000) for the interaction

 RightLeftdummy#c.values

Does it means that the "moderation" is significative in terms that the interaction effect at different values of "values" create differents outputs, that are significative differents (significative moderation) or is significative in how values "affect" differently left ideologies and right ideologies but the "moderation"of each could not necesarily be differents outputs at different values of "values" (significative different left and right but not necesarily significative moderation)?

Hope to be relatively clear with the questions about how I see the two option that the significance of the interaction could be interpreted.

Thank you Clyde again for all your help in this road to obtain my Ph.D (one day).

Sincerely yours,

Alejandro

[Clyde Schechter]

The American Statistical Association has recommended that the concept of statistical significance be abandoned. See https://www.tandfonline.com/doi/full...5.2019.1583913 for the "executive summary" and https://www.tandfonline.com/toc/utas20/73/sup1 for all 43 supporting articles. Or https://www.nature.com/articles/d41586-019-00857-9 for the tl;dr.

So I don't want to advise you as to how to interpret a "significant" interaction term. Let's talk, instead, about something meaningful.

The coefficient of the interaction term here estimates the difference between the outcome:values slope in in the presence of left ideology and the outcome:values slope in the presence of right ideology. (I'm assuming that you are no longer using quadratic terms for values. If you are, then you must also have an interaction term of the form RightLeftDummy#c.values#c.values. In that case, there is really no simple way of interpreting the RightLeftDummy#c.values term by itself.) It tells you the extent to which the outcome:values relationship differs according to the prevailing ideology. The question you need to answer is whether this difference is large enough to be meaningful in a real world sense. Is it large enough to have practical implications? And is the difference estimated precisely enough to make confident answers to those questions? To answer the last of these questions entails looking at the confidence intervals around the interaction coefficient. Look at the endpoints of that interval: if the actual difference in the outcome:values relationship according to ideology were at one of those endpoint values instead of the actually obtained coefficient estimate, would you draw different conclusions? Would you do anything different in response to that? This is the way to think about interaction terms.

[Alejandro Torres]

Dear Clyde,

Thank you very much first for the articles, I downloaded for reading it at night. Second, thank you for the answer, for me is clear and very practical and helpful.
I hope one day to dominate statistics in this great way.
Thank you very much again.

Alejandro.