Confidence intervals for the difference of two regression coefficient means

[Kamelia Georgieva]

Hi all,

I am trying to calculate confidence intervals for the difference of two regression coefficient means.

I have panel data on mututal fund flows, performance and market states and I am running two types of quarterly cross sectional regressions:
- reg flows low_performance
- reg flows high_performance
where low and high performance are just the performance variable split in two.

Then I store the coefficients ( FPS_L and FPS_H ) of each regression and each quarter and average them according to the market states variable (total of 10 market states), which gives me the average flow performance sensitivity of low performance (avg_FPS_L) and high performance (avg_FPS_H) in each market state (group_mkt_rf). Then I managed to calculate the confidence intervals of those averages within each market state using ci mean.


What I am interested in is the difference in sensitivity to low and high performance in each market state, so for each market state I subtract the average sensitivity to high performance from the average sensitivity to low performance. And what I want to calculate next is the confidence interval for this difference.


I tried the following approaches which give me the same result, but I am unsure whether this is the correct result at all. So I would much appreciate some guidance on whether my approach makes (statistical) sense.


Examples for market state 1:

1)

// generate difference between sensitivity to low and high performance
gen FPS_LH = FPS_L - FPS_H
// calculate CI for mean of differences for market state 1
ci mean FPS_LH if group_mkt_rf==1

Output:

    Variable |        Obs        Mean    Std. Err.       [95% Conf. Interval]
-------------+---------------------------------------------------------------
      FPS_LH |          8    .2283137    .3054095        -.493865    .9504923

2)

ttest FPS_L ==FPS_H if  group_mkt_rf==1

Paired t test
------------------------------------------------------------------------------
Variable |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
   FPS_L |       8   -.1396271    .3490874    .9873682   -.9650876    .6858334
   FPS_H |       8   -.3679408    .4586592    1.297284   -1.452497    .7166158
---------+--------------------------------------------------------------------
    diff |       8    .2283137    .3054095    .8638284    -.493865    .9504923
------------------------------------------------------------------------------
     mean(diff) = mean(FPS_L - FPS_H)                             t =   0.7476
 Ho: mean(diff) = 0                              degrees of freedom =        7

 Ha: mean(diff) < 0           Ha: mean(diff) != 0           Ha: mean(diff) > 0
 Pr(T < t) = 0.7605         Pr(|T| > |t|) = 0.4791          Pr(T > t) = 0.2395

[Carlo Lazzaro]

Kamelia:
welcome to this forum.
Some comments about your query:
1) if, as you stated, you're working with panel data, as -flows- is a continuous regressand, your first choice should be -xtreg- rather than -regress-;
2) it looks really weird to me that you have only one predictor in your regression (-performance-). My last memories of corporate finance hide well in the past millennium, but I would find really difficult to defend the results of your simple (ie, with one predictor only) regression even in front of a finance amateurs audience;
3) I'm not clear why you want to peform two different regression when you can compact everything in an one-step approach, just letting -fvvarlist- notation creating a two-level categorical variable for -performance- (say, 0=low; 1=high). This way, you should end up with something like (assuming you will go random effect):

xtreg flows i.performance

Then you can use -test- and/or other postestimation commands to investigate what you're interested in.

[Kamelia Georgieva]

Carlo
thank you so much for your input.

I definitely agree with your comments and am indeed performing exactly what you suggest in a later step in my analysis so let me give some background on the procedure I am using.

1. I am replicating a paper which performs the following analysis:
1.1. A graphical analysis is first performed where cross sectional regressions of flows on performance are run, then averaging the coefficients within each market state group. This analysis should give a first graphical impression of whether investors react stronger to fund performance depending on the market state.
1.2. In the second step, this relationship is tested more formally by running a time fixed effects regression of flows on performance including an interaction with a dummy variable for moderate and extreme market states and further control variables, exactly as you suggested.
Not the issue of this post but this is the code I am using:

reghdfe  flows performance performancexmoderate  `controls' , absorb(quarter) vce(cluster crsp_fundno quarter)

2. I am extending this analysis by allowing for different slopes depending on whether investors react to positive or to negative performance. (The motivation behind this is that in the literature there has been some evidence that the flow performance relationship is not linear, like the model above assumes)
2.1. To obtain a similar graphical representation as in 1.1 from the original paper, I perform the steps that I described in my first post. So in the end I would have the following charts:
- Chart 1: showing the flow performance sensitivity in different market states when investors observe only low performance.
- Chart 2: showing the flow performance sensitivity in different market states when investors observe only high performance, including confidence intervals
- Chart 3: showing the difference between the above sensitivities in each market state
The goal is to split the total effect observed in 1.1 between low and high performance (Chart 1 & 2) and then within each market state compare the sensitivity to low and high performance (Chart 3)
2.2. Afterwards I test the findings by doing a time FE spline regression of flows on performance + moderate market dummy + controls.


So what I am struggling with is the confidence interval in 2.1 Chart 3.


Hope that explains the motivation behind running two separate regressions.




For reference the paper I am replicating is:
Franzoni, Francesco, and Martin C. Schmalz, 2017, Fund flows and market states, Review of Financial Studies 30, 2621–2673.

[Carlo Lazzaro]

Kamelia:
all that I can add to my previous reply is the following toy-example:

. use "C:\Program Files\Stata16\ado\base\a\auto.dta"
(1978 Automobile Data)

. regress price i.foreign

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(1, 72)        =      0.17
       Model |  1507382.66         1  1507382.66   Prob > F        =    0.6802
    Residual |   633558013        72  8799416.85   R-squared       =    0.0024
-------------+----------------------------------   Adj R-squared   =   -0.0115
       Total |   635065396        73  8699525.97   Root MSE        =    2966.4

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     foreign |
    Foreign  |   312.2587   754.4488     0.41   0.680    -1191.708    1816.225
       _cons |   6072.423    411.363    14.76   0.000     5252.386     6892.46
------------------------------------------------------------------------------

. mat list e(b)

e(b)[1,3]
           0b.         1.          
      foreign    foreign      _cons
y1          0  312.25874  6072.4231

. lincom 1.foreign - _cons

 ( 1)  1.foreign - _cons = 0

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         (1) |  -5760.164   1037.715    -5.55   0.000    -7828.811   -3691.518
------------------------------------------------------------------------------

.

[Kamelia Georgieva]

Thank you, Carlo! This example helps me a lot.