pscore: The balancing property is not satisfied

[Jade Li]

Dear Stata Users,

I am runing pscore regression and I am getting an error because one or some of the variuables are not balanced across treatment and control group. Could you help maybe with finding a solution for this?
For example, a variable in sector 1 is unbalanced:

the code I use is:

foreach num of numlist 1(1)28{
use pdata_1, clear
keep if sector==`num'
pscore treatment  v1 v2 v3 v4 v5 v6 v7, level(0.01) pscore(prscore)  blockid(b1) comsup
keep id year prscore b1 comsup
sort id year
save p_res`num', replace
}

the error I get is:

******************************************************
Step 1: Identification of the optimal number of blocks
Use option detail if you want more detailed output
******************************************************


The final number of blocks is 5

This number of blocks ensures that the mean propensity score
is not different for treated and controls in each blocks



**********************************************************
Step 2: Test of balancing property of the propensity score
Use option detail if you want more detailed output
**********************************************************

Variable v5 is not balanced in block 3

The balancing property is not satisfied

Try a different specification of the propensity score

Thank you for your help!

Best
JLi

[Felix Bittmann]

Due to this binary decision of pscore (either it is balanced or not) I find it very frustrating to use. I suggest using kmatch will gives you much more options and insights.

ssc install kmatch, replace

https://www.stata.com/meeting/uk17/slides/uk17_Jann.pdf

[Jared Greathouse]

Or this one.

[Jade Li]

Dear Felix and dear Jared,

I have tried kmatch ad it worked. I also tried psestimate; it worked too. But the problem is that they are giving the same problem for some other sectors in the data. What I did was that changing the definition of sectors in my data.

Thank you for your replies.

Best wishes
Ali

[Asya Saydam]

Hello Ali,

I am having the same problem with pscore (the balancing property not being satisfied) and I haven't tried kmatch or postestimate since I need to apply psm but I was wondering if you find a solution to pscore too? What did you mean by changing the definition of sectors?
I would appreciate it if you can share what you did.

Thank you!
Asya.

[Jade Li]

Hi Asya,

I tried to merge similar industries together. So for example industries like "COKE AND REFINED PETROLEUM PRODUCTS", and "CHEMICALS AND CHEMICAL PRODUCTS" have codes 19 and 20, respectively (according to the OECD classifications). So in, say, industry 20, the number of observations is too little and the balancing property is not satisfied, then you merge both industries together to have the balancing property satisfied. You can always change the way you identify and classify the industries in your data, but of course, they must be related to a similar category. You cant merge 20 with 10 (FOOD PRODUCTS AND BEVERAGES). I also think you have to mention that in your paper. This is what I did.

I hope this helps!
Jade Li

[Felix Bittmann]

PSM in general cant work wonders. If the data are too unbalanced there is no way to get a "good" result with it (other methods like OLS will fail as well, but more in silence). One solution is to coarsen categories, which can also be done using CEM, which is available in kmatch.