[Carlo Lazzaro]

Clyde's reply is correct and mine on -T_C- was wrong.
I forgot the interaction is my example, that was exactly what Matthew and Scott were asking about.
Thanks as usual to Clyde and sorry for any misguidance from my side.

[Clyde Schechter]

Here, I recognize the A group coefficient from T_C 6 years, in that people from group A after 6 years are 1.9 percentage points more likely to be active than people from group A after 3 years.

Correct.

I'm so sorry that I have another question related to the interpretation of agent##T_C: R## 6 years: coefficient of .038 --> This means that group R after 6 years is 3.8pp more likely to perform an action compared to R after 3 years. Is this correct?

Also correct.

I'm so sorry that I have another question related to the interpretation of agent##T_C: R## 6 years: coefficient of .038 --> This means that group R after 6 years is 3.8pp more likely to perform an action compared to R after 3 years. Is this correct?


Q: If so, Why does this table not reflect the same coefficients of 0.037 for R and -.021 for S. (taken from agent##T_C) - but instead has different coefficients? I see how the margins table shows each group after 6 years compared to 3 years. So why are these coefficients not included? Where can I find the coefficients 0.037 for R and -.021 for S in a table?

Good catch! I made a mistake in my previous post. My apologies. To fully see what's going on, we have to go back to the underlying algebra.

outcome = cons + b_T_C * (TC = 6 years) + b_agent_R *(agent = R) + b_agent_S*(agent = S) 
+ b_T_C#agent_R*(TC = 6 years & agent = R) + b_T_C#agent_S*(TC = 6 years & agent = S) + error

So let's consider now the case of agent = R and contrast T_C = 6 years vs. T_C = 3 years. So we have agent = R is 1 and agent = S is 0. For the T_C = 3 years situation, all of the underlined terms are 0. Moreover, since we are dealing with an agent = R case, the b_agent_S*(agent = S) term is also zero. So for agent = R and T_C = 3 years we get outcome = cons + b_agent_R*1.

Now consider the case of agent = R and T_C = 6 years. Of the underlined terms, the one in blue remains zero, because agent = S is false, but the other two are now active. So, for agent = R and T_C = 3 years we get outcome = cons + b_agent_R*1 + b_T_C*1 + B_T_C#agent_R*1.

When we next calculate the difference between those, the b_agent_R*1 term and the constant term are common to both and they cancel. So the difference we are left with is:
b_T_C + b_T_C#agent_R (I'm eliding the *1 since multiplying by 1 changes nothing.) So the marginal effect of 6 years (vs 3 years) conditional on agent = R is the sum of those two coefficients, not just the interaction coefficient. So if you add up the coefficient of T_C 6 years and the interaction coefficient of T_C 6 years and agent R, that will equal the marginal effect that Stata reported.

I'm not showing the arithmetic here because the model you show in #13 differs from the model you originally showed in two critical ways: a) it has additional covariates, which can alter both the coefficients and the results of -margins-, and, more important, b) it does not contain an interaction term. Anyway, if you rerun the regression with the interaction included, and then run -margins agent, dydx(T_C)-, you will get results that agree with my coefficient calculations here as applied to the regression coefficients.

I'm sorry for the confusion I caused with my error in the earlier post. But I'm glad you found the problem and asked the right question!

[Matthew Berg]

Thank you so much! I highly appreciate the detailed response. Do you think it's sufficient to report the interaction from the regression agent, T_C, agent#T_C, or do I need the average marginal effects dydx(agent), dydx(T_C) as well?

I'm so sorry, I promise this is the last question: is there a way to compare the different agents in different T_C --> so e.g., R after 3 years vs. A after 6 years along with a t-stat / p-value? The pwcompare addition you posted was very helpful, but I'm not sure how significant the results are. If I want to report results, the requirements are for this paper, it is supposed to be like in an academic research paper.

Thank you so much, Clyde! I am very very grateful!

[Clyde Schechter]

I'm so sorry, I promise this is the last question: is there a way to compare the different agents in different T_C --> so e.g., R after 3 years vs. A after 6 years along with a t-stat / p-value?

Run -pwcompare agent#T_C, effects

which will give you the output from -margins agent#T_C, pwcompare- along with t-stats and p-values.

When you are writing an academic research paper, there is presumably some particular research question that was investigated. You should report most prominently the statistics that directly answer that question. If the research question was, to what extent, if any, does the effect of 6-years depend on the agent group, then reporting the interaction coefficients from the regression would be the most appropriate. If the question was what are the expected probabilities of action under certain specific (or all) combinations of agent and T_C, then the -margins- output is the most appropriate. If the question was to compare certain expected probabilities of action for certain combinations of agent and T_C, then the corresponding rows from the -pwcompare- output are the direct answer. And so on. Now, that doesn't mean that you discard all the other analyses. But it does mean that the other analyses are presented in some secondary way, or are discarded. They might be included in ancillary tables in an appendix or on-line supplement to the article. Or, they might be included as later tables in the main body of the paper itself. But it is the statistics that answer the research question that should be "in the spotlight" in the paper.

[Matthew Berg]

Thank you so very much, Clyde! I appreciate you!