Which test is the most convincing?

[Doug Bujakowski]

Brief background: I’m examining mediation rates in China. I have a panel dataset with N=24 provinces and T=30 years (1985-2014). For each province-year, I observe mediation rates and host of economic/demographic information.

Anecdotal reports suggest that in 2006 or soon thereafter, the Chinese government began bolstering its mediation system and encouraging its use. My goal is to test this assertion. Unfortunately, I’m unable to quantify the “effort” the government exerts in promoting mediation.

What is the most convincing way to test this assertion? My ideas are listed below. Please evaluate. For all ideas, assume that I am regressing mediation rates on variables thought to influence mediation rates.

1. Include a time polynomial (such as year and year^2) in the regression. If year is negative and year^2 is positive, this is evidence of a parabolic trend, even after controlling for other factors. Proceed by determining whether the minimum of the parabola is around year 2006.
2. Include a lagged dependent variable in the regression, that is, a lag of the mediation rate. Perhaps the best measure of “effort” is the previous year’s mediation rate. After controlling for other factors, I can determine whether this effort proxy is positive and significant.
3. Include year fixed effects. After these effects are estimated, graph their magnitudes against time. Progressively increasing year fixed effects after 2006 would indicate more effort.
4. Include a linear time trend with a kink at 2006. Determine whether the post 2006 trend is significantly larger than the pre 2006 trend. Unlike the other ideas, this one assumes that we know where the change occurs.

Is there any reason to use a combination of these ideas? Do you have other ideas? Please suggest. Thanks for your help!

[Carlo Lazzaro]

Doug:
at their face-value, I would sponsor strategies 1) or 3).

[Doug Bujakowski]

Hi Carlo. Thank you for the response. I posted this same question on another research forum and a couple people suggested that I should test for a structural break in the data. I can't seem to figure out whether structural break tests are appropriate for my purpose. My understanding is that a structural break refers to a sudden change in the mean or variance of a variable. I'm not proposing that the mean or variance of mediation rates changes immediately at 2006, nor do I have reason to believe that the betas on my regressors change at 2006. Rather, my hypothesis is that beginning in 2006, mediation rates are set on a "new trajectory." For example, declining mediation rates before 2006 and rising rates after 2006 would be consistent with my hypothesis. In this case, means, variances, and betas before and after 2006 could be the same.

Do you happen to know if structural break tests can address my hypothesis of a change in trajectory? I've looked at several papers but am having trouble answering this question on my own.

[Carlo Lazzaro]

Doug:
unfortunately, I cannot say.

[Matthias Stoetzer]

Hi Doug,
have a look at Stata Time Series Reference Manual Release 15 the commands "estat sbsingle" (p. 185) and "estat sbcusum" (p. 173). Estat sbsingle is a test of structural breaks with unknown
break date; estat sbcusum is a test of parameter stability. As far as I understand your problem is a gradual break and in this regard both tests will give you information.

Hope this helps,

Matthias

[Doug Bujakowski]

Hi Matthias. Thank you for the reply. Both of the commands you propose look promising, but both are meant to be used with time series data. I have a panel dataset, i.e. cross-sectional time series. As a result, neither command can be applied to my situation. Do you happen to know if similar structural break tests exist (within Stata) for panel data?