Using rifhdreg to estimate Quantile TE

[Sandra Macedo]

Hi all,
I am studying the effect of a policy on students' scores. Since results are heterogeneous, I thought I could try to estimate unconditional QTE , using a DiD framework. The data is repeated cross-section (N=10 M) and treatment (eligible) is not binary, with 4 treatment groups and never treated=0. In "before" (policy=0) no one is treated, in "after" the 4 treatment groups are treated at the same time. I am also using covariates with and without fixed effects. I wonder if rifhdreg would be appropriate, given the size of the dataset and considering a non-binary treatment. Most methods require a binary variable. A good thing about rifhdreg is that it calculates the IPW which I couldn't do with a categorical variable.


The OLS FE results are quite different (see below)

reghdfe scores i.eligible##i.policy i.maeduc sex age c.age##c.age i.location ymid federal i.popsize munic_gdp, abs(state_school) vce(robust)
(MWFE estimator converged in 1 iterations)
note: age omitted because of collinearity

HDFE Linear regression Number of obs = 10,492,959
Absorbing 1 HDFE group F( 23,10492909)= 210336.66
Prob > F = 0.0000
R-squared = 0.3706
Adj R-squared = 0.3706
Within R-sq. = 0.3432
Root MSE = 0.7934

-----------------------------------------------------------------------------------------
| Robust
scores | Coefficient std. err. t P>|t| [95% conf. interval]
------------------------+----------------------------------------------------------------
eligible |
group 1 | -.4104479 .0036849 -111.39 0.000 -.4176701 -.4032256
group 2 | -.5835029 .0016248 -359.13 0.000 -.5866874 -.5803185
group 3 | -.5346714 .0053164 -100.57 0.000 -.5450914 -.5242514
group 4 | -.7244366 .0015726 -460.67 0.000 -.7275188 -.7213545
|
1.policy | .0328963 .0013969 23.55 0.000 .0301585 .0356342
|
eligible#policy |
group 1#1 | -.0073701 .0045681 -1.61 0.107 -.0163234 .0015833
group 2#1 | -.0523011 .0017602 -29.71 0.000 -.0557511 -.0488512
group 3#1 | -.0177454 .0064512 -2.75 0.006 -.0303895 -.0051014
group 4#1 | -.0310596 .0016786 -18.50 0.000 -.0343496 -.0277696
|
maeduc |
Primary | .0600287 .0015395 38.99 0.000 .0570114 .063046
Middle school | .102138 .0015253 66.96 0.000 .0991485 .1051274
High school | .2354342 .0015262 154.26 0.000 .2324429 .2384254
University | .4406134 .0017124 257.31 0.000 .4372572 .4439696
Postgrad | .5003833 .0018582 269.29 0.000 .4967413 .5040252
|
sex | -.0815129 .0005098 -159.88 0.000 -.0825122 -.0805137
age | -.6658851 .0014303 -465.55 0.000 -.6686885 -.6630817
age | 0 (omitted)
|
c.age#c.age | .0138486 .0000339 408.30 0.000 .0137822 .0139151
|
location |
rural | -.1727199 .0014851 -116.31 0.000 -.1756306 -.1698092
ymid | .0177454 .0000427 415.28 0.000 .0176616 .0178291
federal | 1.133364 .0017353 653.14 0.000 1.129963 1.136765
|
popsize |
medium | .1244144 .0006323 196.77 0.000 .1231751 .1256536
large | .2168877 .0007244 299.40 0.000 .2154678 .2183075
|
munic_gdp | -2.25e-10 3.45e-12 -65.33 0.000 -2.32e-10 -2.18e-10
_cons | 7.663446 .014959 512.30 0.000 7.634127 7.692765
-----------------------------------------------------------------------------------------

Absorbed degrees of freedom:
------------------------------------------------------+
Absorbed FE | Categories - Redundant = Num. Coefs |
--------------+---------------------------------------|
state_school | 27 0 27 |
------------------------------------------------------+

I guess this is probably because of the propensity scores.

Now when I try rifhdreg effects are positive.

rifhdreg scores i.eligible##i.policy, rif(q(50)) over(policy) abs(state_school) rwlogit(i.maeduc sex age c.age##c.age i.location ymid federal i.popsize munic_gdp) att trim(0.01,0.98)

HDFE Linear regression Number of obs = 10,492,959
Absorbing 1 HDFE group F( 9,10492923)= 184767.80
Prob > F = 0.0000
R-squared = 0.1535
Adj R-squared = 0.1535
Within R-sq. = 0.1259
Root MSE = 1.1772

-----------------------------------------------------------------------------------------
| Robust
scores | Coefficient std. err. t P>|t| [95% conf. interval]
------------------------+----------------------------------------------------------------
eligible |
group 1 | -.3530913 .0052722 -66.97 0.000 -.3634247 -.3427579
group 2 | -.9244422 .0022359 -413.46 0.000 -.9288244 -.9200599
group 3 | -.4669798 .0080754 -57.83 0.000 -.4828072 -.4511523
group 4 | -1.150093 .002121 -542.24 0.000 -1.15425 -1.145936
|
1.policy | -.1039593 .0016711 -62.21 0.000 -.1072346 -.1006841
|
eligible#policy |
group 1#1 | .1458918 .0062365 23.39 0.000 .1336685 .1581151
group 2#1 | .0613481 .0024609 24.93 0.000 .0565247 .0661714
group 3#1 | .1575793 .0094327 16.71 0.000 .1390915 .176067
group 4#1 | .0880934 .0022803 38.63 0.000 .0836241 .0925627
|
_cons | .7040373 .0015276 460.87 0.000 .7010432 .7070314
-----------------------------------------------------------------------------------------

Absorbed degrees of freedom:
------------------------------------------------------+
Absorbed FE | Categories - Redundant = Num. Coefs |
--------------+---------------------------------------|
state_school | 27 0 27 |
------------------------------------------------------+
Distributional Statistic: q(50)
Sample Mean RIF q(50) : -.08672


I then use the qregplot command for each group
qregplot 2.eligible#1.policy , q(5(5)95) ols raopt( color(black%5))



The graphs look cool (I couldn't paste here) but I am not sure whether I am doing the right thing and how to compare the results with OLS. I also wanted to put the results for each model into one graph. Can anyone help?
Thanks for the help!!

[Sandra Macedo]

What I really want to know is whether I can trust these results or whether my code is wrong.

[FernandoRios]

Hi Sandra
Sorry for the delay answering.
So, I think your code is fine with one caveat. You mention your treatment variable is not binary but has multiple levels. However, your variable policy seems to be binary. Im guessing that you were referring to multiple treatments because of eligibility?

The way things are currently written (in your code), you are using an average treatment effect on the treated, with binary treatment asl well. The fact that you use reweighting means that you are trying that those not affected by the policy to look like those who were affected by it. So on that regard, can be considered as Quantile treatment effect.

If you do include a multivalued treatment, you will have to use rwmlogit (or rwmprobit). The problem, however, is that there is no "ATT" equivalent, but only a ATE.

Let me know if you have other questions either here or by email (look in the helpfile)
Best wishes
F

[Sandra Macedo]

Hi FernandoRios , thanks a lot for helping out and for your time. I realised the original code was wrong. In fact, I shouldn't have used policy in the over option, because policy is merely the time variable (before x after) and since I can't use a multilevel treatment (I want the att because it's a DiD design) I used a binary treatment variable ("treat") instead.

Here is my code.

rifhdreg scores treat##i.policy2, rif(q(50)) over(treat) abs(state_school) rwlogit(i.maeduc sex age c.age##c.age i.race i.location ymid federal i.popsize munic_gdp) att trim(0.01,0.98)

HDFE Linear regression Number of obs = 9,852,136
Absorbing 1 HDFE group F( 3,9852106) = 41464.75
Prob > F = 0.0000
R-squared = 0.0778
Adj R-squared = 0.0778
Within R-sq. = 0.0572
Root MSE = 1.1371

-------------------------------------------------------------------------------
| Robust
scores | Coefficient std. err. t P>|t| [95% conf. interval]
--------------+----------------------------------------------------------------
1.treat | -.5810692 .0038284 -151.78 0.000 -.5885727 -.5735657
1.policy | -.0502511 .0041038 -12.24 0.000 -.0582945 -.0422077
|
treat#policy |
1 1 | .0142762 .0042129 3.39 0.001 .006019 .0225335
|
_cons | .2654408 .0037228 71.30 0.000 .2581442 .2727374
-------------------------------------------------------------------------------

Absorbed degrees of freedom:
------------------------------------------------------+
Absorbed FE | Categories - Redundant = Num. Coefs |
--------------+---------------------------------------|
state_school | 27 0 27 |
------------------------------------------------------+
Distributional Statistic: q(50)
Sample Mean RIF q(50) : -.06905

However, I realise that for QTE weighting should only be done only for the period before the treatment. Wonder if there's a way to solve that.

I sent an email to [email protected] with other enquiries.
thanks so much!

[Sandra Macedo]

Answering your question. Yes treatment is based in four different eligibility criteria, but I realise I can't estimate the effects for each one separately using a weighting method.