State Specific Linear time trends

[sanjay nawaz]

Dear friends,

I am running a difference-in-difference design on effect of labour market law in India. Data I have is panel data at worker level, and am interested in how worker hour changed after law implementation. Data I have is on workers in two state, and one state implemented the law while other state did not implement the law. Rajasthan having the law, and Gujarat not having the law. The regression I have is:

 xtset worker year
gen law= (year >=2002) 
xtreg hours i.rajasthan##i.law i.year, fe vce(cluster state)

My FE will be worker FE. Is this the correct specification?

Am now wanting to implement state specific linear time trend. I have looked at the forum and decided to implement following regression:

xtset worker year
gen law= (year >=2002) 
xtreg hours i.rajasthan##i.law i.year c.year##i.rajasthan, fe vce(cluster state)

I have 3 questions:

1. Include the specific linear trend for gujarat state which was not treatment state as well? So I will be having the regression:

xtset worker year
gen law= (year >=2002) 
xtreg hours i.rajasthan##i.law i.year c.year##i.rajasthan c.year##i.rajasthan c.year##i.gujarat , fe vce(cluster state

)

2. Which variable should be omitted in each specifications? I am seeing STATA omitting my i.law and i.rajasthan variable in first specification, and many more in second specification. and am not understanding why.

3. What is the meaning behind the 'year' variable? This is the time trend before law implementation between two states?


My model is similar to this famous model: http://econ.lse.ac.uk/staff/rburgess/wp/indreg.pdf

on top of page 18, but I have data at worker and not industry level.

Many thanks,

Sanjay

[Clyde Schechter]

First, I have a question. Within each state, do you have true panel data? That is, are the same workers being followed both before and after 2002? If so, your first specification looks correct to me.

As for your second model:

xtset worker year
gen law= (year >=2002)
xtreg hours i.rajasthan##i.law i.year c.year##i.rajasthan, fe vce(cluster state)

the only thing I would question is whether you really want to have both i.year and c.year in the same model. It's not illegal; it isn't necessarily wrong. And I know that in economics there is a tendency to throw in fixed effects for everything in sight. But your yearly shocks (i.year) and c.year are competing with each other to model the time trends, with the result that you are probably not getting a clear picture of either. And, in particular, it will make your attempt to tease out the different time trends in the two cities a nightmare at best, impossible at worst. I would leave i.year out. But this is a matter of taste and judgment. If you decide you really do want both yearly shocks and linear time trends in your model, then I would clone the year variable and then use the clone for the time trend: if you don't give them separate names Stata will get confused when it tries to separate things out. Also, you can't have all of the usual year indicators in your model along with continuous time (regardless of the name) because there is colinearity. I think that in this situation Stata's default is to omit the last colinear variable mentioned in the command. So to assure that the time trend stays in, you should put it before the year indicators. You will lose two year indicators: the usual reference category and one other that Stata will select (usually, I think, the last).

clonevar time_trend = year
// OMIT INDICATORS FOR 1997, 1998 SO TIME TREND VARIABLE IS RETAINED
xtreg hours i.rajasthan##i.law c.time_trend##i.rajasthan i.year, fe vce(cluster state)

As for your third model:

xtset worker year
gen law= (year >=2002)
xtreg hours i.rajasthan##i.law i.year c.year##i.rajasthan c.year##i.rajasthan c.year##i.gujarat , fe vce(cluster state)

this makes no sense. With c.year##i.rajasthan you already are modeling separate time trends in rajasthan and gujarat. Adding in the gujarat variable provides nothing additional and, in fact, Stata is going to drop one of the two states and its corresponding interaction term from the model due to colinearity. I'm not sure which one it will drop. But since rajasthan + gujarat = 1, those are colinear, and similarly rajasthan#c.year + gujarat#c.year = c.year, so there is colinearity there as well. As for which of the interactions, or c.year itself, will be dropped, I cannot say. But at least one of the three will go. So stick with your second model. This third one is just going to get you confused.

I've already explained some of the variables that will get omitted from the model due to colinearity in your third model. But let's also go back to your second model. Because it is a fixed effects model, and each worker's value of the rajasthan variable remains fixed over time, the rajasthan main effect will be omitted. That's perfectly fine: it is not a problem because it isn't needed for anything, and wouldn't be estimable if it were. In addition to the usual 1 reference year that gets omitted to avoid colinearity among all the levels of year, you will lose another year due to colinearity with i.law. So altogether, there will be two omitted year indicators.

Since you were tempted to include additional terms for gujarat, I'm inferring that you don't know how to interpret the results of the second model so as to find, for example, the gujarat time trend in the output. The easiest way to do this is with the -margins- command. After your second model (the first one shown in my response) I would run:

margins rajasthan#law, noestimcheck // EXPECTED HOURS IN EACH CITY BEFORE AND AFTER
margins rajasthan, dydx(year), noestimcheck // TIME-TRENDS IN EACH CITY

[sanjay nawaz]

Thank you Clyde, that was very insightful.

I am not understanding the margins command. What will this be showing?

Am also wanting to compute the residuals after my diff-in-diff estimation by putting in "rvfplot" but STATA replies with "last estimates not found".

What is the problem here?

thanking you

[sanjay nawaz]

Am wondering about different specification now.

If I am wanting to do consider the effects of subgroup, then should I delete non-subgroup observations from dataset?
Or is it better to put in sub-group dummy. I am asking because I would like to test whether policy effect would be different for different age groups.
I am thinking of the following implemention:

 
 xtreg hours i.rajasthan##i.law c.year##i.rajasthan YOUNG, fe vce(cluster state)

Where YOUNG is my age dummy. Is this best way, or should I cancel observations of everyone not YOUNG?

Also Clyde thanking you for discussion, I have managed to remove i.year from specification as I think c.year is better at capturing effects.

[Clyde Schechter]

In my view, the best explanation of the -margins- command is Richard Williams' Stata Journal article: http://www.stata-journal.com/sjpdf.h...iclenum=st0260.

\Just including YOUNG will not do it: it will tell you the difference in expected hours among the young and the old, but it will not tell you about whether YOUNG modifies the effect of law. For that you need an interaction between law and young. So I would add i.YOUNG##i.law to the model.

[sanjay nawaz]

Thanking Cylde for the explanation.

I have included i.YOUNG##i.law to the specification. But am wondering whether this is becoming triple differences model.
If this is triple difference model, my STATA code implementation will be:

 
 xtreg hours i.rajasthan##i.law##i.YOUNG c.year##i.rajasthan YOUNG, fe vce(cluster state)

This estimate is treating the treatment group as Rajasthani Youth and not just all worker in Rajasthan. Am now perplexed on following. Conceptually what is difference between doing this triple difference model,

 xtreg hours i.rajasthan##i.law##i.YOUNG c.year##i.rajasthan, fe vce(cluster state)

and deleting all observations who are not YOUNG and then implementing

 xtreg hours i.rajasthan##i.law c.year##i.rajasthan, fe vce(cluster state)

Am saying that i can define group of interest by simply deleting all observations of workers who do not belong to group of interest. After this I can run simply diff-in-diff model. Which approach is should be implemented?

[sanjay nawaz]

Apologies on mistake. My first coding is wrong in previous post. It should be

 
 xtreg hours i.rajasthan##i.law##i.YOUNG c.year##i.rajasthan, fe vce(cluster state)

[Clyde Schechter]

Sorry, this is getting a bit complicated and confusing. I was partly confused because I mis-remembered your variables and though law was the variable distinguishing the treatment and control groups. But it is not. rajasthan is your treatment vs. control group variable and law is your time variable. So if you want to know whether the effect of the law is different in the young and old, then you do, indeed, regress on the three way interaction i.YOUNG##i.rajasthan##i.law. And of course you can still include the c.year##i.rajasthan time trends as well. The additional question arises whether you think that the time trends also differ by YOUNG. In that case you might want to have a c.year##i.rajasthan##i.YOUNG interaction instead of just the two way interaction. Either way, I daresay that you will have a difficult time interpreting the results of this unless you use the -margins- command afterward. Adding combinations of interaction coefficients is just too complex when there are this many floating around. (It's the kind of thing you should do just once in your life as a student to verify that you understand the concept, but she leave to automation when doing it "in real life.")

Am saying that i can define group of interest by simply deleting all observations of workers who do not belong to group of interest. After this I can run simply diff-in-diff model.

This is not a good idea. While you can run separate models, one on the sample with YOUNG = 1 and the other on the sample with YOUNG = 0, you will be left with results that you have no way to compare with each other. So you will get an estimate of the effect of the intervention among the YOUNG and a separate effect of the intervention among the not YOUNG. You can subtract those to get a point estimate of the difference between those effects, but you will not have any confidence bounds on the difference between them. A point estimate without an interval estimate around it is usually not much use.

[If you were using OLS regression instead of fixed effects, a viable alternative would be to run the separate YOUNG = 0 and YOUNG = 1 regressions, and then compare the results using -suest-. But -suest- does not support -xtreg, fe-, so that route is closed off.]

[sanjay nawaz]

Sorry for limited explanation.

Thank your for the very useful tip of c.year##i.rajasthan##i.YOUNG.

If I delete all non-YOUNG observations and run the simple diff-in-diff, will my model not be capturing the difference of law on youth between Rajasthan and Gujarat?
In this case, what is advantage of using both YOUNG=1 and YOUNG=0 instead of just using the YOUNG=1?

One more thing, maybe If I have law which is applicable only to 18-25 year old so YOUNG is treated group. My data is for everyone working from 18+. Then is it better to use triple difference model, and use all observations approach, or use delete all non-YOUNG observations and do simple diff-in-diff?

My research design at the moment is that I have potentially two laws to use (they will be different studies, but I have identified two laws and I am deciding on which law and study I will undertake).

[Clyde Schechter]

If I delete all non-YOUNG observations and run the simple diff-in-diff, will my model not be capturing the difference of law on youth between Rajasthan and Gujarat?

Yes, it will. But you will not be able to then compare that to the effect of law on non-youth between Rajasthan and Gujarat. If you don't want/need to do that, then just running it on the YOUNG = 1 sample is fine.

There is, actually, a slight advantage of keeping both groups in the model, using the interaction in the regression, and then relying on -margins- to calculate the specific marginal effect among the young. A problem that is encountered with subgroup analyses in general is that the subgroups are smaller than the whole sample, often much smaller, and so the effect estimates calculated for them in a separate subgroup-specific analysis are noisier. By using the entire sample and a model with interactions, some of that noise gets smoothed out. This is sometimes referred to as "borrowing from strength" (although people who dislike this approach sometimes call it "borrowing from weakness!") or "regularization." So my usual approach to determining effects in subgroups is to keep all the data and run models with interactions. But just restricting a non-interacted model to the subgroup in question is also fine if there is no need to draw contrasts among different subsets.

All of the above remarks are, of course, applicable only if the treatment/control distinction applies to all of the subgroups in question. With regard to your second law that applies only to 18-25 year olds, then, definitely you should analyze only those for which YOUNG = 1. In fact, for such a law, analyzing the entire data set with an interaction model really would not make any sense, as it would be asking for an estimate of the effect of the law on a group that is "not in universe" for that law, and for which the effect of the law is, a priori zero. (OK, there might be some "spillover" effect on the older, non-covered group, depending on what your outcome is and what the law does. But presumably that is not what you are looking for.)

[sanjay nawaz]

Thanking you again Clyde, you are most helpful

There is some more complexity in my design. There is also some ambiguity in the law because it is technically only applying to the big companies. I say technically because it is not stated explicitly in the law, but it is quite clear from its implementation. So the small firm workers might be affected, but I think they have too many other more important issues that affects them for them to be worried about the law change (I know this because my father runs small firm!). So one might say there is leakage from my treatment group, but i don't think the leakage is that important. Because as you say putting in too many interactions is not very good modelling, so am thinking not doing the 4 difference model where treatment group is YOUNG worker in Rajasthan working in BIG firm. Plan is to conduct 3 difference model which treatment affect is given by [YoungRajasthanPOST-YoungRajasthanPRE]-[YoungGujaratPOST-YoungGujaratPRE]-[OldRajasthanPOST-OldRajasthanPRE], so am wondering where will this small firm effect get captured? Or is the recommendation that I drop all observation of SMALL firms because they are not technically affected by the law, and then run the triple difference model with the treatment group YOUNG worker in Rajasthan. In this case is there advantage of keeping SMALL firm? Is there another way to approach. maybe run diff-in-diff only with SMALL firms and show it is not important?

Also am wondering how to plot residuals after the diff-in-diff regression.

thanking you Clyde

[Clyde Schechter]

so am wondering where will this small firm effect get captured? Or is the recommendation that I drop all observation of SMALL firms because they are not technically affected by the law, and then run the triple difference model with the treatment group YOUNG worker in Rajasthan.

Well, it certainly sounds like you have good reason to believe that small and large firms will show different responses to this law. So what I would say strongly is what you should not do. Don't use the entire data set while not including a firm size#rajasthan interaction term.

Whether you should further complicate your model by throwing in yet another interaction, or just exclude the small firms from the data set is a bit harder to say here. I'm not an economist/econometrician; I'm an epidemiologist. Perhaps the analogous thing in my area is that we usually assume that things work so different in children and adults that we study them separately, or analyze their data separately. Your situation sounds somewhat analogous to that, and my instinct would be to analyze just the large firms, and then perhaps later separately analyze the small firms. But I'm not sure how well epidemiologic instincts carry over to your situation. I think this is a question you should direct towards a colleague in your discipline for better advice.

The only purely statistical consideration I see here is that separate analyses will not enable you to formally compare and contrast the results in the small firms with those in the large firms; only an interaction based approach can do that. So if you think that small vs large comparison is important, then you should use the interaction approach from the outset. From what you wrote about "maybe run diff-in-diff only with small firms and show it is not important," I sense that you see this small vs large comparison as, at most, a secondary issue. That would incline me to keep the model simple and analyze the large firms only.

[sanjay nawaz]

Thank you Clyde.

I have decided to implement triple difference approach and not include small firms in data-set. My specification is now:

xtreg hours i.rajasthan##i.law##i.YOUNG c.year##i.rajasthan, fe vce(cluster state)

My question is now conceptually, the diff-in-diff-in-diff estimate will be [YoungRajasthanPOST-YoungRajasthanPRE]-[YoungGujaratPOST-YoungGujaratPRE]-[OldRajasthanPOST-OldRajasthanPRE] which I believe is correct coming from bottom of page 2 of the following lecture notes:

http://www.nber.org/WNE/lect_10_diffindiffs.pdf

Making sure that this diff-in-diff-in-diff specification will tell me "what is the difference in health status among YOUNG in the two states" and it removes two confounding trends:

1. Removes the any trend between the young within the states (classic diff-n-diff)

2. And this also now removes the trend affecting OLD workers in treatment state (Rajasthan).

Am not sure why number two is confounding effect as it say in lecture notes.Why would 2 affect my estimate of the effect of law on YOUNG in the two states? Maybe it's because there is chance that some work trend is common across all worker in treatment state and this term is capturing this effects.

Then am now confused as to justification of adding c.year##i.rajasthan in the model because this is also controlling for time trend in WORK in the two states, but then is this not hugely correlated with what [OldRajasthanPOST-OldRajasthanPRE] will be picking up? Both are picking up this trend in work hours across workers but [OldRajasthanPOST-OldRajasthanPRE] is picking up on old workers, and c.year##i.rajasthan picking up on all workers. Am thinking that maybe c.year##i.rajasthan is informative about different time trend between state but [OldRajasthanPOST-OldRajasthanPRE] is time trend only on old within state. Is this providing much useful informations?

thanking you clyde

[Clyde Schechter]

My question is now conceptually, the diff-in-diff-in-diff estimate will be [YoungRajasthanPOST-YoungRajasthanPRE]-[YoungGujaratPOST-YoungGujaratPRE]-[OldRajasthanPOST-OldRajasthanPRE] which I believe is correct

That looks right to me.

Making sure that this diff-in-diff-in-diff specification will tell me "what is the difference in health status among YOUNG in the two states"

Your outcome variable is called hours. What does this have to do with health status? Of anybody?

Am thinking that maybe c.year##i.rajasthan is informative about different time trend between state but [OldRajasthanPOST-OldRajasthanPRE] is time trend only on old within state.

The inclusion of c.year##i.rajasthan will adjust for linear time trends that may occur within Rajasthan and within Gujarati. The model assumes that such time trends would be the same for both young and old. If that is not the case, then you need to replace that with a three way interaction i.YOUNG##c.year##i.rajasthan. Whether the assumption that the time trends would be the same for both young and old is a reasonable one, I cannot say. That is a substantive issue and I don't know the content and have no expertise here.

but then is this not hugely correlated with what [OldRajasthanPOST-OldRajasthanPRE] will be picking up? Both are picking up this trend in work hours across workers but [OldRajasthanPOST-OldRajasthanPRE] is picking up on old workers, and c.year##i.rajasthan picking up on all workers.

I don't understand this.

[sanjay nawaz]

Thank you Clyde

I will be incorporating i.YOUNG##c.year##i.rajasthan in my regression. What I am saying with my last point is that if I implemented the diff-in-diff-in-diff without the i.YOUNG##c.year##i.rajasthan term what will I NOT be controlling for. The confusion is because my diff-in-diff-in-diff estimate will be the effect of the law of young workers in Rajasthan after removing the parallel trend in workers in Gujarat and the trend in old workers in rajasthan ( [YoungRajasthanPOST-YoungRajasthanPRE]-[YoungGujaratPOST-YoungGujaratPRE]-[OldRajasthanPOST-OldRajasthanPRE] ). What new thing is the i.YOUNG##c.year##i.rajasthan term capturing? In my opinion this helps capture and control for different pre-treatment trends between the two states. Am I right? But then it will also capture post treatment trends between the two groups, but is this not what I want my diff-in-diff-in-diff parameter want to capture?