How to export p-value after using -synth2- command (for synthetic control)

[Noah Spencer]

I am running a synthetic control analysis using the user-created "synth2" command. I am running a unit-level placebo test and want Stata to extract the p-value generated by this test automatically.

As an example, in the following analysis, I would want to extract the value 0.2000 from the output "Note: The probability of obtaining a post/pre-treatment MSPE ratio as large as 1's is 0.2000" and store it in a matrix:

* Load data
webuse nlswork.dta, clear

* Keep subset of data to ensure balanced panel
keep if (year == 73 | year == 75 | year == 77 | year == 78)
keep if (idcode == 1 | idcode == 2 | idcode == 5 | idcode == 6 | idcode == 9)

* Declare panel
tsset idcode year

* Run analysis
synth2 ln_wage ln_wage(75) ln_wage(73), trunit(1) trperiod(77) placebo(unit)

* Set up matrix to store result in 
mat pval = J(1,1,.)

* Store result (with help from Statalist community)...

If there's a simple way to do this automatically, that'd be great. Alternatively, I may have to calculate the p-value myself. Doing so would require some matrix manipulation that I'm not sure how to code. I will try to explain the process I need help completing.

The following output matrix lists the ratio of post-treatment MSPE to pre-treatment MSPE in column 3:

mat li e(mspe)

What I need to do is calculate the ordinal rank of the treated unit's ratio and divide it by the number of units studied. In this case, the ratio for the treated unit (1) is the largest of the 5 ratios, so it gets rank 1. If we then divide it by the total number of units studied (5), we get the desired 0.2000. If anyone knows how to write code to tell Stata to do this, it'd be much appreciated!

Thanks!

[George Ford]

Looks like the pvals are stored in e(pval).

can extract using, say,

local p1 = el(e(pval),2,1)

often, I store results like that in a new matrix which shortens up pulling values later on.

matrix P = e(pval)

[George Ford]

I'm sure MATA would do it better and I'm not sure exactly what you're after, but this may help.

matrix P = J(5,3,.)
local c = 1
foreach tr in 1 2 5 6 9 {
    synth2 ln_wage ln_wage(75) ln_wage(73), trunit(`tr') trperiod(77) placebo(unit) nofig
    matrix P[`c',1] = `tr'
    matrix P[`c',2] = el(e(pval),1,4)
    matrix P[`c',3] = el(e(mspe),1,3)
    local c = `c'+1
}
matrix list P
capture drop P*
svmat P
egen rankit = rank(P3), field
tab rankit if P1==1

[Noah Spencer]

Thanks George. Unfortunately, e(pval) seems to only store the p-values for each post-treatment period individually, rather than considering them collectively as I would like to do. (The 0.2000 number showing up in the e(pval) table here is not generalizable to other cases, including my own research.)

[George Ford]

And I see the placebo does the placebo stuff automatically. But I don't think that's what you want here.

e(att) has the average effect for all periods.

matrix P = J(5,2,.)
local c = 1
foreach tr in 1 2 5 6 9 {
    synth2 ln_wage ln_wage(75) ln_wage(73), trunit(`tr') trperiod(77) nofig
    matrix P[`c',1] = `tr'
    matrix P[`c',2] = `e(att)'
    local c = `c'+1
}
matrix list P
capture drop P*
svmat P
capture drop prob
egen prob = rank(P2), field
replace prob = prob/5
tab prob if P1==1

[Noah Spencer]

Thanks George!