F tests

Sumedha Gupta

Join Date: May 2016
Posts: 278

14 Nov 2019, 09:03

Dear All,

I have a weekly panel data.

Code:


	Code:
	* Example generated by -dataex-. To install: ssc install dataex
clear
input float ID int userTRA float(week f event_time_dateweek)
1  0  1 . -6
1  0  2 1 -5
1  0  3 1 -4
1  0  4 2 -3
1  0  5 2 -2
1  0  6 3 -1
1  0  7 3  0
1  0  8 4  1
1  0  9 4  2
1  0 10 5  3
1  0 11 5  4
1  0 12 6  5
1 69 13 6  6
2  0  1 . -6
2  0  2 1 -5
2  0  3 1 -4
2  0  4 2 -3
2  0  5 2 -2
2  0  6 3 -1
2  0  7 3  0
2  0  8 4  1
2  0  9 4  2
2  0 10 5  3
2  0 11 5  4
2  0 12 6  5
2  0 13 6  6
3  0  1 . -6
3  0  2 1 -5
3  0  3 1 -4
3  0  4 2 -3
3  0  5 2 -2
3  0  6 3 -1
3  0  7 3  0
3  0  8 4  1
3  0  9 4  2
3  0 10 5  3
3  0 11 5  4
3  0 12 6  5
3  0 13 6  6
4  0  1 . -6
4  0  2 1 -5
4  0  3 1 -4
4  0  4 2 -3
4  0  5 2 -2
4  0  6 3 -1
4  0  7 3  0
4  0  8 4  1
4  0  9 4  2
4  0 10 5  3
4  0 11 5  4
4  0 12 6  5
4  0 13 6  6
5  0  1 . -6
5  0  2 1 -5
5  0  3 1 -4
5  0  4 2 -3
5  0  5 2 -2
5  0  6 3 -1
5  0  7 3  0
5  0  8 4  1
5  0  9 4  2
5  0 10 5  3
5  0 11 5  4
5  0 12 6  5
5  0 13 6  6
6  0  1 . -6
6  0  2 1 -5
6  0  3 1 -4
6  0  4 2 -3
6  0  5 2 -2
6  0  6 3 -1
6  0  7 3  0
6  0  8 4  1
6  0  9 4  2
6  0 10 5  3
6  0 11 5  4
6  0 12 6  5
6  0 13 6  6
7  0  1 . -6
7  1  2 1 -5
7  0  3 1 -4
7  0  4 2 -3
7  0  5 2 -2
7  0  6 3 -1
7  0  7 3  0
7  0  8 4  1
7  0  9 4  2
7  0 10 5  3
7  0 11 5  4
7  0 12 6  5
7  0 13 6  6
8  0  1 . -6
8  0  2 1 -5
8  0  3 1 -4
8  0  4 2 -3
8  0  5 2 -2
8  0  6 3 -1
8  0  7 3  0
8  0  8 4  1
8  1  9 4  2
end

I want to test if I should collapse the panel to fortnightly or whether the weekly data adds predictive power to the model. To do so I run:

Code:

reg userTRA i.week if week>1 , vce(cluster ID) /*regression 2 - unrestricted;  R-square
> d         =     0.0020 OR R-squared         =     0.0001 (balanced sample)*/

Linear regression                               Number of obs     =    854,496
                                                F(11, 71207)      =      11.38
                                                Prob > F          =     0.0000
                                                R-squared         =     0.0001
                                                Root MSE          =     .21174

                                (Std. Err. adjusted for 71,208 clusters in ID)
------------------------------------------------------------------------------
             |               Robust
     userTRA |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        week |
          3  |    .000323   .0011896     0.27   0.786    -.0020085    .0026545
          4  |  -.0015729   .0011263    -1.40   0.163    -.0037804    .0006346
          5  |  -.0027665   .0010688    -2.59   0.010    -.0048614   -.0006716
          6  |  -.0017414   .0011155    -1.56   0.119    -.0039278    .0004451
          7  |  -.0046905   .0010685    -4.39   0.000    -.0067847   -.0025963
          8  |  -.0041428   .0010874    -3.81   0.000    -.0062741   -.0020114
          9  |  -.0082715   .0010086    -8.20   0.000    -.0102484   -.0062947
         10  |  -.0023031    .001104    -2.09   0.037    -.0044669   -.0001393
         11  |  -.0041709   .0010941    -3.81   0.000    -.0063153   -.0020264
         12  |   -.004199   .0010748    -3.91   0.000    -.0063056   -.0020923
         13  |  -.0039883   .0014575    -2.74   0.006     -.006845   -.0011317
             |
       _cons |   .0299405   .0008242    36.33   0.000     .0283251    .0315558
------------------------------------------------------------------------------




. reg userTRA i.f if f~=., vce(cluster ID) /*regression 2 - restricted; R-squared       
>  =     0.0011 OR R-squared         =     0.0001 (balanced sample)*/

Linear regression                               Number of obs     =    854,496
                                                F(5, 71207)       =      16.17
                                                Prob > F          =     0.0000
                                                R-squared         =     0.0001
                                                Root MSE          =     .21175

                                (Std. Err. adjusted for 71,208 clusters in ID)
------------------------------------------------------------------------------
             |               Robust
     userTRA |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           f |
          2  |  -.0023312   .0007826    -2.98   0.003     -.003865   -.0007974
          3  |  -.0033774    .000772    -4.37   0.000    -.0048906   -.0018643
          4  |  -.0063687    .000748    -8.51   0.000    -.0078347   -.0049026
          5  |  -.0033985   .0007798    -4.36   0.000     -.004927     -.00187
          6  |  -.0042551   .0009093    -4.68   0.000    -.0060374   -.0024728
             |
       _cons |    .030102    .000596    50.51   0.000     .0289338    .0312701
------------------------------------------------------------------------------

How can I get the required F test to check if the 'unrestricted' model with the weekly panel is jointly significant?

Similarly, if I want to compare two models with the second model having an additional interaction term what is the right way to request an F-test on the joint significance of the interaction term?

Code:

. xtreg userTRA post event_time_dateweek  if (event_time_dateweek>-63 & event_time_datewe
> ek<6), fe vce(robust)

Fixed-effects (within) regression               Number of obs     =    854,496
Group variable: ID                              Number of groups  =     71,208

R-sq:                                           Obs per group:
     within  = 0.0000                                         min =         12
     between =      .                                         avg =       12.0
     overall = 0.0000                                         max =         12

                                                F(2,71207)        =      19.06
corr(u_i, Xb)  = 0.0000                         Prob > F          =     0.0000

                                       (Std. Err. adjusted for 71,208 clusters in ID)
-------------------------------------------------------------------------------------
                    |               Robust
            userTRA |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
--------------------+----------------------------------------------------------------
               post |   -.001768   .0008099    -2.18   0.029    -.0033554   -.0001806
event_time_dateweek |  -.0001345   .0001181    -1.14   0.255     -.000366    .0000969
              _cons |   .0274946   .0003891    70.66   0.000      .026732    .0282572
--------------------+----------------------------------------------------------------
            sigma_u |  .06329344
            sigma_e |  .19674718
                rho |  .09378464   (fraction of variance due to u_i)
-------------------------------------------------------------------------------------

.
end of do-file

. do "C:\Users\Sumedha\AppData\Local\Temp\STD00000000.tmp"

. xtreg userTRA post event_time_dateweek post#c.event_time_dateweek if (event_time_datewe
> ek>-63 & event_time_dateweek<6), fe vce(robust)

Fixed-effects (within) regression               Number of obs     =    854,496
Group variable: ID                              Number of groups  =     71,208

R-sq:                                           Obs per group:
     within  = 0.0001                                         min =         12
     between =      .                                         avg =       12.0
     overall = 0.0001                                         max =         12

                                                F(3,71207)        =      15.18
corr(u_i, Xb)  = 0.0000                         Prob > F          =     0.0000

                                         (Std. Err. adjusted for 71,208 clusters in ID)
---------------------------------------------------------------------------------------
                      |               Robust
              userTRA |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
----------------------+----------------------------------------------------------------
                 post |  -.0027966   .0008746    -3.20   0.001    -.0045109   -.0010823
  event_time_dateweek |   -.000325   .0001401    -2.32   0.020    -.0005996   -.0000504
                      |
                 post#|
c.event_time_dateweek |
                   1  |   .0007238   .0002626     2.76   0.006     .0002092    .0012385
                      |
                _cons |   .0269231   .0004502    59.80   0.000     .0260407    .0278056
----------------------+----------------------------------------------------------------
              sigma_u |  .06329344
              sigma_e |  .19674642
                  rho |   .0937853   (fraction of variance due to u_i)
---------------------------------------------------------------------------------------

I will be grateful for your help.
Sincerely,
Sumedha.

Tags: None

Phil Bromiley

Join Date: Apr 2014

Posts: 4348
#2

15 Nov 2019, 09:32

You didn't get a quick answer. Your post is extremely long - a shorter post is more likely to get a response.

To your first question, with r-square of .0001, your model is not explaining anything - the significant coefficients come from an immense data set. The first two models have rmse of .21174 and .21175 so there is no substantive difference between the models. While an F might be significant due to the large sample size, it doesn't matter. Both models are equally poor.

For the second question, the parameter significant on the interaction indicates it is statistically significant - you don't need to run with and without to do this test. While there is a way to test using explained variance, it isn't needed given the t statistic. But, again, neither model explains much of anything.
Comment
Sumedha Gupta

Join Date: May 2016

Posts: 278
#3

15 Nov 2019, 09:55

Thank you Prof. Bromiley for your clear and helpful response. I apologize for the very long post - was trying to be clear, but obviously did not get the balance right.

I understand that the tiny R-squares make it clear that none of the models are great. However, abstracting from that, if I wanted to carry out the test to see if the weekly dummies have joint predictive power beyond the restrictive specification with the fortnight dummies [basically imposing 3.week=4.week and 5.week=6.week etc.], what would be the stata code to get a joint F-test?

Thank you again.
Sincerely,
Sumedha.
Comment

Announcement

F tests

Comment

Comment