inteff and margins command after Non-Linear Difference in Differences Model

[Abu Tauheed]

Dear All,

I have a data which satisfy the assumptions of Athey and Imbens(2006) "Non-Linear Differences in Differences Model: Inference and Interpretation. They suggested the changes in changes model for discrete dependent outcome variables. Lastly, they suggested if the model satisfy all the assumption the interpretation for the interaction term can be done by co-efficients or odd ratio. Furthermore, while referring the literature for non-linear model interaction term ex. Puhani(2013) Norton(2004) Richard(2012), I have two options, either to use "margins" command or use "inteff" command advocated by Richard(2012).

My model is like

 xtlogit Y i.Time i.Treatment i.Treatment#i.Time i.urban i.Treated_State, or nolog

For the analysis I have "strongly balanced panel data", further I have matched the individuals on set of observable characteristics using propensity score, suggested by Heckman(1978) and Rubin(1995) and Athey & Ibmens (2006). Moreover, all the variables are code with 0 and 1, where both has meaning. Here we have two time period i.e. pre and post, moreover the Treated_State is a factor variable identifying the states which received special treatment.

The objective of the present study is to predict the Pr(Y=1) for the individuals who received treatment overtime. In addition, I also want to estimate the treatment effect (interaction term) over urban and treated state, to see how the treatment effect varies in between Treated State and Non-Treated State in rural and urban areas.

Therefore, to see the marginal effect ( I want to calculated conditional marginal effect ) of the interaction term only I have used the following command.

margins Treatment#Time, level(95)
marginsplot, ylin(0) noci

Furthermore, to see the conditional marginal effect of the interaction over Treated State and Rural region, the following command is used;

margins Treatment#Time, at (urban=(0 1) Treated_State=(0 1)) level(95)
marginsplot, ylin(0) noci

My Question: Have I correctly written the margins command to see the TREATMENT EFFECT with time period over Treated Stated in rural and urban areas. I am confused in using the above command, or shall use the following command-

margins, dydx(JSY) at (Period=(0 1) Treated_State=(0 1) urban=(0 1)) level(95)
marginsplot, ylin(0) noci

In addition, I have also tried the inteff command. in STATA 13.0 for my xtlogit model, it seems that the command doesn't support my model.

[Clyde Schechter]

margins Treatment#Time, level(95)
marginsplot, ylin(0) noci

will display and plot the expected values of Y in each treatment group at each time. It does not calculate marginal effects.

To get the marginal effect of treatment in the treatment group you want:

margins Treatment, dydx(Time) pwcompare(cimargins effects)

This will give you the difference in probability of Y in each group as time passes from the pre-treatment to the post-treatment era, and it will also calculate the difference between those (which is your DID estimator of treatment effect in the probability metric) and give you a standard error, confidence intervals, and test of the null hypothesis that the difference is zero.

Your code

margins Treatment#Time, at (urban=(0 1) Treated_State=(0 1)) level(95)

will give you marginal outcomes in the treatment and time groups conditional on four combinations of urban and Treated_State. Again, there are no marginal effects here.
Added: The above code could be simplified to:

margins Treatment#Time#urban#Treated_State

Finally, your code

margins, dydx(JSY) at (Period=(0 1) Treated_State=(0 1) urban=(0 1)) level(95)

should produce nothing other than an error message, as there is no JSY variable in your regression.

The clearest explanation of the -margins- command that I know if is Richard William's excellent Stata Journal article: http://www.stata-journal.com/sjpdf.h...iclenum=st0260. I think if you read that you will be more comfortable using -margins-. It doesn't have all of the detail of the manual chapter, but it covers the most frequently applications of the command and explains in very simple terms what is going on.

[Abu Tauheed]

Dear Sir,

Thank you very much for your quick reply.

JSY was a typing error, however by JSY I meant Treatment variable. Moreover, the

marginsplot

command is no more working for graphical representation.

In addition, if instead of double interaction I have triple interaction where my model is as follows:

xtlogit SAFEM i.Period i.JSY i.JSY#i.Period i.Treat_State i.Treat_State#i.Period i.JSY#i.Treat_State#i.Period i.URBAN, or nolog

where SAFEM is dependent variable, JSY is treatment variable and Treat_State indicates the treated or non-treated states along.

Now if I want to calculate the marginal effect of treatment in the treatment group with three interaction term; where Treat_State indicates whether the State is Treated or non-treated State.
Shall I use the following command:

margins JSY, dydx(Period Treat_State) pwcompare (cimargins effect)

If I further want to see how the this triple interaction term different for urban and rural areas which command shall I use, because the policy was intended to improve the outcome in rural areas with especial reference to Treated States ( where Treat_State is "0" if non-treated state and "1" if treated state)

Lastly, I have two sets of independent variable controlling for supply side and demand side variables, shall I estimate the marginal effect command after controlling these variables.

[Clyde Schechter]

First, let's simplify the coding of the model:

xtlogit SAFEM  i.JSY##i.Treat_State##i.Period i.URBAN, or nolog

does the same thing and is a lot easier to reada.

If I further want to see how the this triple interaction term different for urban and rural areas

If that is what you want, you need a different model with a four way interaction:

xtlogit SAFEM  i.JSY##i.Treat_State##i.Period##i.URBAN, or nolog

I don't know what you mean when you say that -marginsplot- is "no more working." Are you getting error messages? If so, what are they? If no error messages, are you getting outputs that don't seem to make sense to you? If so, show what they are and explain what you don't like about them.

Do remember that -marginsplot- can only be run immediately after a -margins- command. If you run any other commands after -margins-, then you lose the ability to run -maginsplot-.

Now if I want to calculate the marginal effect of treatment in the treatment group with three interaction term; where Treat_State indicates whether the State is Treated or non-treated State.

I can't answer because it is not clear to me which marginal effects you want to calculate here. In a three way interaction there are many marginal effects: each of the main effects has marginal effects conditional on each combination of the values of other two variables. If I assume that each of the variables JSY, Treat_State and Period is simply dichotomous, that's 12 different marginal effects. And that doesn't include the possibility of getting averaged marginal effects over some of those. So you have to be very explicit what you want here. (In fact, I would not be surprised to learn that you have not yourself thought through which one(s) you want and that is your main obstacle to coding it. If you can be explicit about which marginal effects you want, you may find that you don't even need any help figuring out how to code for them.)

[Abu Tauheed]

Sir,

I want to calculate the marginal effect of treatment on the treated group initially with two, further the interaction goes upto for 4 variables.

Therefore, I am looking for the Average Marginal Effect (AME) and Marginal Effect at Representative(MER).

I am little confused when my interaction term increase.

My Question:

1. How to modify the command for marginal treatment effect on the treated, with three and four interaction term. For interaction with two variables you suggested the following i.e.

 
 margins Treatment, dydx(Time) pwcompare(cimargins effects)

2. How should I calculate the Marginal Effect at Representatives (MER) with upto four interaction term.

[Abu Tauheed]

When I using command for marginsplot, Stata pop ups "term not labeled r(182)"

[Carlo Lazzaro]

Abu:
is there any variable in -string- format included in your code?

[Abu Tauheed]

No Sir, there is no string format variable.

[Carlo Lazzaro]

Abu:
try to insert a comma before -pwcompare(cimargins effects)-.

[Clyde Schechter]

I am looking for the Average Marginal Effect (AME) and Marginal Effect at Representative(MER).
... How should I calculate the Marginal Effect at Representatives (MER) with upto four interaction term.

I'm a little bit confused, because over the course of this thread the variable names have switched a bit and I'm not entirely sure what corresponds to what. I'm operating here on the assumption that what we originally called Treatment is now called JSY, and what we originally called Time is now called Period. I take it you first want an estimate of the average marginal effect of treatment (by a DID estimator. In that case, the code is actually the same as suggested before:

margins JSY, dydx(Period) pwcompare(cimargins effects)

The DID estimator of the average treatment effect will be found in the contrast betweeen the marginal effect of Period for JSY = 0 and the marginal effect of Period for JSY = 1. The averaging is implicit in the absence of any specification for the other variables.

Next you say you want marginal effect at representative. I'm not sure what representative values you might have in mind because, from their names, I would guess that both URBAN and Treat_State are dichotomous variables. So each of those variables has only two possible values, and there are therefore only four possible combinations. I don't quite get how you would decide which among these four combinations are "representative" and which are of no interest. In any case, I'll leave that issue to you. What you can do is:

margins URBAN#Treat_State#JSY, dydx(Period) pwcompare(cimargins effects)

The output from this will be copious with all possible combinations of URBAN, Treat_State and JSY, (8 of them) and all possible pairs of those (64 in all). Most of these will not be answers to this specific problem. But in that list you will find, for example 1.URBAN#1.Treat_State#1.JSY vs 1.URBAN#1.Treat_State#0.JSY. That contrast is the DID estimate of the marginal effect of treatment at the values URBAN = 1 & Treat_State = 1. The same idea applies to any interesting values of URBAN and Treat_State. Basically, any contrasts of the form x.URBAN#y.Treat_State#1.JSY vs x.URBAN#y.Treat_State#0.JSY (where x takes the same values on both sides of vs, and so does y) will be a DID estimate of the marginal effect of treatment at the values URBAN = x and Treat_State = y.

Yes, multi-way interactions can be very confusing to work with. If you think it's complicated doing it with -margins-, just try to imagine how difficult it was before we had margins and we had to figure out exactly what predictions to calculate and how to average them together!

[Abu Tauheed]

Dear Sir,

Thank you very for your reply, I have much clarity on the margins command now. Moreover, you have understood my model correctly.

My "CONFUSION" was on how much interaction I must introduce in my non-linear DiD model to estimate the Average Marginal Effect of Treatment correctly. This was mainly because, apart from calculating the AME of treatment (between JSY and Period) I am also keen to see how the treatment effect varies Between Treated and Non-Treated States and within their urban and rural regions. Thus my objective is to calculate the following:

1. The Average Marginal Effect of Treatment for JSY and Period.
2. To see how this treatment effect varies for the representative values of Treat_State ( Treat-State take two values 0 and 1)
3. Further, I want to see how the treatment effect varies for urban and rural areas both within and between treated and non-treated states.

Thus I will estimate the following xtlogit model:

xtlogit Y i.Period##i.JSY i.Treat_State i.URBAN

For AME Treatment effect on the treated

margins JSY, dydx(Period) pwcompare (cimargins effect)

For Treatment Effect at representatives value of Treat_State and URBAN I have the following command:

margins JSY#Treat_State#URBAN, dydx(Period) pwcompare(cimargins effect)

Note: My marginsplot command is not working after these command, Stata pop ups the error that "term not labeled- r(182)"

[Clyde Schechter]

This looks good to me.

As for "My marginsplot command is not working after these command, Stata pop ups the error that "term not labeled- r(182)," I have never encountered that problem before and I don't know what to say about it. Can you post a small example data set that reproduces the same problem?

[Claire McKenna]

I'm doing an analysis using a DiD model, with a binary outcome (probability of becoming employed, from unemployment), so I'm using a logit model. I cluster standard errors at the state level, and control for state and year_month fixed effects. Clyde, I'm using your suggested syntax to be able to isolate the probability of reemployment in the treatment and control groups, in the pre- and post-periods, and the difference between them:

margins JSY, dydx(Period) pwcompare(cimargins effects)

. But it doesn't seem to work when my model incorporates state and year_month fixed effects. It generates the tables, but with periods where estimates should be, and "(not estimable)" in SE columns. But it works when I exclude them. Why is that? What seems to be the problem?

[Clyde Schechter]

In order to try to answer this question, I would need to see the exact commands and complete output (including any messages) of the -logit- regressions themselves, both with and without the state and year-month fixed effects.

[Claire McKenna]

Thanks. Here's the code I use, w/ and w/o fixed effects. This is a separate but similar analysis to a DiD (I'm comparing differences in groups in a pre-period and a post-period) (I could upload the output as an attachment; would that work?)

// w/o fixed effects
logit reemp3 i.hid##i.postperiod_nc b3.age_group b1.race_wbho b4.edu4 i.woman##i.marstdum1##i.ownkidd_18 b1.ind_nilf b1.uh_occmaj_b2 sampjl b1.durg ur_sa ur2_sa ur3_sa iur iur2 iur3 initrate initrate2 initrate3 empgrowth emp2 emp3 l_incrate_jhu stringd if cutoff3==1 & sampall==1 & age>=18 & age<65 [pw=wtfinl], vce(cluster statefip) or
margins hid, dydx(postperiod_nc) pwcompare(cimargins effects)

// w/ fixed effects

logit reemp3 i.hid##i.postperiod_nc b3.age_group b1.race_wbho b4.edu4 i.woman##i.marstdum1##i.ownkidd_18 b1.ind_nilf b1.uh_occmaj_b2 sampjl b1.durg ur_sa ur2_sa ur3_sa iur iur2 iur3 initrate initrate2 initrate3 empgrowth emp2 emp3 l_incrate_jhu stringd i.year_month i.statefip if cutoff3==1 & sampall==1 & age>=18 & age<65 [pw=wtfinl], vce(cluster statefip) or
margins hid, dydx(postperiod_nc) pwcompare(cimargins effects)