Test mean difference between two periods with bootstrapping

[Daniel Novak]

Dear all,

I'm using group-wise regressions (regression for each industry) to estimate earnings management proxies for 2017 - 2020. The proxies are the standard deviations of the residuals from the regression per year (variable name: DACC).
Now I want to test the significance of the differences for the mean of DACC for 2017-2019 and DACC for 2020 by using bootstrapping.
In the literature the procedure is described as follows.

The difference between the two periods was tested with a bootstrapping procedure using 10000 simulations and 100 randomly extracted observations to calculate our proxies 10000 times

or

The significance of these differences was tested using a bootstrapping procedure that randomly chooses a sample of 100 firms-year observations with replacement from a period. We computed our attributes of earnings management and repeated the procedure 10,000 times. We next performed an independent-samples t-test

(please refer also to page 14 and 31 in this paper)

Due to the group-wise regression I don't know how to code this bootstrap procedure in Stata and would be grateful for help.

My code for the group-wise regression:

local max=r(max)

forvalues i = 1/`max' {

 di "Busy with regression " `i' " of " `max'
 qui reg lag_tacc inv_ta term2 lag_gppe mtbv lag_cfo if sic2id == `i', const    
 qui predict res if sic2id == `i', res
 qui replace DACC = res if sic2id == `i'
 qui drop res
 
}

My dataex:

* Example generated by -dataex-. For more info, type help dataex
clear
input int firm_id byte country_id int year float(lag_tacc inv_ta term2 lag_gppe) long mtbv float(lag_cfo sic2id DACC)
 1  6 2017   -.21149993  .00002904022   -.08087701   .5703209  121   -.06827356 81   -.1921046
 1  6 2018   -.28330782  .00003828484    .07909647   .5980092  252    .16428025 82   -.2039483
 1  6 2019    -.3702169  .00005115613    .27383876   .9244424  434     .2076427 83  -.24963287
 1  6 2020   -.16377135  .00003657243      .083312    .672567  388    .07921588 84  -.19028614
 4 12 2017    -.4426975  .00002230102    .06676925  .05396847  405   -.02016012 81   -.4085054
 4 12 2018    -.1366944  .00003731761   -.12221517  .05586446  249   .034481473 82   -.0910366
 4 12 2019  -.066249944  .00004735521     .0855235  .07998295  191  -.028744614 83  .006397666
 4 12 2020   -.09596388   .0000385862    .09669702  .21068066  203   .031949375 84  -.11472715
 8  6 2017   -.12694831  .00005247966    .11676726  .05594332  339    -.4482288 89  -.12997842
 8  6 2018   -.03124413  .00002087116 -.0005009079  .02462797  292    -.3373197 90  -.08304719
 8  6 2019   -.03935244 .000031993855   -.06629127   .1067635  205    -.4073458 91  -.15237276
 8  6 2020   -.06169622  .00001888467  .0020017752  .06330142  212    -.2968859 92 -.068773516
10 12 2017   -.03886048  9.163047e-06  -.008878993  .06744919   31   -.17219198 33 -.037153263
10 12 2018   -.01437592 .000011418523  -.013199813   .0908686   15    -.2870046 34  -.04350368
10 12 2019   -.04879713 .000017061935  -.016498892  .16932264  293    -.4357789 35   -.1683069
10 12 2020    -.0965563 .000027682427    .07244491   .2744159  606    -.7064002 36   -.1970784
12 12 2017   .009532467 .000012284107     .6862025  .29469573 1713    .29791415 81   .09988267
12 12 2018   .011322294 .000011660448     .8658349  .31110075 1092    .33232275 82   .09112692
12 12 2019   -.04012904 .000010973215    1.8842217   .4021573 2048     .4725944 83   .22041237
12 12 2020  .0030978306  8.440956e-06    -2.514485   .3870347  151   .019371994 84   .04584527
15 12 2017    .03710942  .00001170275 -.0020596841  .20526624   39   -.00995904 53  .024740925
15 12 2018    .02774003 .000010373982   .016162664  .17135744   75   .004014731 54  .011322052
15 12 2019   .013900174 .000012500156    .04252553   .1042388   75   .021325266 55   .05073176
15 12 2020   -.16165225   .0000208934   .027349465   .3977059   57    .15494546 56 -.036724627
16 12 2017    -.2003822  2.773456e-06   .034049716  .14982486 1208     .2306184 81   -.1387806
16 12 2018    .13258937  3.039089e-06    -.0246227  .18210524  507     .1086687 82    .1783426
16 12 2019    -.1089102 3.0190774e-06    .17074695  .29631945  489     .2247824 83  -.02919579
16 12 2020     -.377012  2.611682e-06     .4966166   .2542394  958     .4025229 84  -.06681383
17  6 2017    -.1502375  7.153485e-06   -.07281532  1.0180125    8    .02311291 25  -.09570345
17  6 2018   -.07149943  7.689764e-06   -.04473905  1.1271349   42   .025537707 26  -.02528145
17  6 2019   .035469312  7.821238e-06    .05732185  1.1810772   61  -.016729627 27  -.03276328
17  6 2020   -.04408404  7.934493e-06   .008632729  1.1778834    6    .05561286 28  -.00259405
19 29 2017   .005451973  5.642112e-07    .19891886   .7031363  388    .06307091 17    .0304972
19 29 2018   .013381124  5.845328e-07   .007477928     .76346  349    .06286709 18  .024582025
19 29 2019  -.003809967  5.638549e-07   .002765708   .8367685  437    .08360163 19  .035912476
19 29 2020   -.02112117  4.613825e-07   .027126985   .6947895  437    .09288645 20  .019093165
20 22 2017   -.04628849  3.514691e-07     .0603121    .809574  315    .11816393 49  -.02711454
20 22 2018  -.028870394  3.453396e-07   .016541768   .8404186  194     .1111303 50   .02443839
20 22 2019   -.04855082  3.192033e-07   .030962717   .8668603  244      .120595 51  .005732412
20 22 2020    -.0754296  2.897795e-07   -.06189689   .7655394  227    .10942073 52  -.01483942
23  6 2017   -.06073205 .000015990534   .013336105  .25682396  119   -.08751619 65  -.01107045
23  6 2018   -.03487282  .00002038154  -.005136149   .3299568   88   -.12137209 66   .03140805
23  6 2019    -.3345639 .000024611143    .04656428   .4184633   68   -.14385213 67  -.24182725
23  6 2020    -.2052736  .00004050387   -.10101665   .5465592   93   -.15670946 68  -.11299095
24 24 2017    .04873721  2.417281e-06     .5961474  .14368075   69  -.009429812 69 -.001261443
24 24 2018  .0018217107 2.2490256e-06   -.13378105  .13150053   33    .03009871 70   .03776131
24 24 2019   .000637391  2.213163e-06     .2858278  .13210592   49   .029439494 71   .01353492
24 24 2020   -.09432708 2.0711652e-06     .2469326  .12301686   52    .12448531 72  -.07380601
25 12 2017    .09342242   .0000400096   -.06713611   .5218853  508     .0824998 65    .1534517
25 12 2018   -.05551094  .00002509536     .3544971   .4252911  277     .2516312 66  -.04106335
25 12 2019  -.002412981  .00001899985     .3449802   .4590743  458    .19655342 67   .08862455
25 12 2020   -.00268527  6.920799e-06   -.00997979   .2275074  454    .03787753 68   .08283819
28 12 2017   -.07270404   2.66794e-06    .11608207  .14105934  616     .2000128 33   .00541529
28 12 2018  -.018384174 2.4998876e-06     .0244514   .1244994  582    .17714202 34  .020728264
28 12 2019  -.066463776 2.2085392e-06    .06748633   .1516935  554     .1842739 35 -.003742309
28 12 2020   -.10715587 1.9376841e-06   -.03203961   .4303558  699    .13421562 36 -.037128355
42 22 2017   .003084374 1.3611535e-06    .04160774  .27088043  206     .0112091 61 -.006608323
42 22 2018  -.034673095 1.4248242e-06    .05136064  .29219297  155     .0635557 62  -.02456995
42 22 2019     .1236878 1.3171448e-06 -.0007547233  .25153512  214   -.08206075 63    .0883061
42 22 2020   -.16997238 1.2000394e-06     .2682172  .23893143  184    .24776973 64 -.027592055
45  8 2017  -.016511265 6.0937076e-08    .06445857   .7391178  104   .029925406 13  .009833422
45  8 2018  -.018824024  6.119206e-08    .02971254   .7752207  129    .03889685 14  .026297644
45  8 2019  -.025770916   7.06703e-08   -.04853106   .9848959   15    .05062411 15   .01955432
45  8 2020  -.035614446 6.0649135e-08    -.0518145   .8731359  188    .05867944 16  .032259133
48 12 2017 -.0000860824   9.56471e-06    .10761255   .3448078  102    .07956883 73  .018400755
48 12 2018    .04447161  8.794069e-06   -.02875661  .34008425   84    -.2400693 74  -.09773506
48 12 2019   -.17750525  9.057315e-06   -.06007717   .4289001  134    .05193464 75  -.11504456
48 12 2020   -.20063645  9.250608e-06    .14176558  .50942177  229    .18431836 76 -.017394267
50  8 2017   -.03083736 2.3384844e-07    .14519136  1.1020449  173    .08559157 45 .0033750276
50  8 2018  -.021124577  2.362162e-07    .09662116  1.1613593  116    .07712813 46  .001415558
50  8 2019    -.0937166 2.2393186e-07   -.04989963  1.1459218  145    .08038259 47   .00962612
50  8 2020   -.08649358 2.3278746e-07  -.014792246    1.28269  155    .09791157 48  -.01342812
52  2 2017  -.014691986  7.848787e-08   .023571633   .3653912  164    .03843692 13 .0027491925
52  2 2018  -.011583562  7.485146e-08     .0178636   .3757524  139   .033263464 14  .016674543
52  2 2019   -.01219385  7.113809e-08   -.00871655   .3827228  135    .04028628 15  .011955497
52  2 2020  -.022215927 6.5830434e-08   -.05788127   .3618442  116    .03734317 16  .016931392
55  8 2017  -.034172803 3.2193967e-08    .08722193  .14830737  274    .05999269 13 -.017997153
55  8 2018   -.03806817 3.3514574e-08    .07406929   .1530024  243   .068734705 14 .0021331897
55  8 2019    -.0431921  3.048817e-08    .04131025   .2015132  254    .07252254 15  .004079641
55  8 2020   -.01555919 2.7408236e-08    -.0818678  .13289656  239   .031291652 16  .006916937
57 11 2017  -.016177343 2.8391266e-06   -.02372658   .3022534  111    .03963989 57  .003426753
57 11 2018    .02421821  2.478327e-06   .010746025   .3277513   51 -.0018463536 58  .006911682
57 11 2019   -.06240558 1.9825143e-06    .11964077    .361565   62    .07946313 59  -.00723695
57 11 2020   -.07189625 1.8017304e-06   -.14746262   .3277221   47     .0375859 60  -.05757552
60 24 2017   -.14616814  5.279497e-06   -1.5695838   .3184064   76    .07481574 69  -.00546426
60 24 2018   -.08426762  6.614413e-06   -.12374244   .3848133    9    .04774944 70 -.037041575
60 24 2019   .016191393  7.641054e-06    .12097316   .4474066  109  -.025192555 71  .029034203
60 24 2020    .27575958  7.780587e-06      .509504   .4195215   47    .04015561 72    .2299776
63 12 2017  -.030995617  .00007827176   -.06191296  .05220727 1025    -.6212429 81  -.11998166
63 12 2018   -.04110521  .00004250797     .0133475  .02746015  942    -.3004463 82  -.08162448
63 12 2019   -.11691304  .00006399181    .09131631   .1125616  953    -.3295578 83  -.02390794
63 12 2020   -.06803017  .00007252683 -.0015230635  .11234406  793    -.3112852 84    -.507347
64 22 2017   -.08031105 .000031869462     .3281917   .0535407  497    .09879533 81 -.008936507
64 22 2018    .04581121  .00002589667    .09905476 .033121843  308   -.03941473 82    .0663601
64 22 2019   -.12951274  .00002524615     .0940419  .09073466  366    .16364555 83  -.05970334
64 22 2020   .006401525 .000022700446    .28775084  .06830564  533     .0767729 84  .020475514
65 14 2017   -.04326066  5.661649e-06    .13620794   .9390807   95    .09665567 21   .04209612
65 14 2018    -.0916216  5.511737e-06    .14301305  1.0085431   94    .15807663 22 -.020576086
65 14 2019  -.037181307  4.750391e-06     .0856638   .9806184   93    .10270344 23  .012030553
65 14 2020  -.072298706 4.4281683e-06   -.17226905   .8715034   78    .09982863 24    .0773303
end
format %ty year

[Carlo Lazzaro]

Daniel:
welcome to this forum.
Do you mean somethng along the following lines?

. g year_cat=0 if year<=2019

. replace year_cat=1 if year>2019 & year!=.

. label define year_cat 0 "2017-2019" 1 "2020"

. label val year_cat year_cat

. ttest DACC, by( year_cat) unequal

Two-sample t test with unequal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. err.   Std. dev.   [95% conf. interval]
---------+--------------------------------------------------------------------
2017-201 |      75   -.0200602     .010941    .0947516   -.0418606    .0017401
    2020 |      25   -.0402973    .0261384     .130692   -.0942443    .0136497
---------+--------------------------------------------------------------------
Combined |     100   -.0251195    .0104542    .1045418   -.0458629   -.0043761
---------+--------------------------------------------------------------------
    diff |             .020237    .0283358               -.0374247    .0778988
------------------------------------------------------------------------------
    diff = mean(2017-201) - mean(2020)                            t =   0.7142
H0: diff = 0                     Satterthwaite's degrees of freedom =    32.82

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) = 0.7599         Pr(|T| > |t|) = 0.4802          Pr(T > t) = 0.2401

 . bootstrap r(t), reps(1000) strata(country_id) cluster(firm_id) saving(Daniel_bootstrap, every(1) double replace) : ttest DACC, by( year_cat) un
> equal nodots

Bootstrap results

Number of strata = 9                                     Number of obs =   100
                                                         Replications  = 1,000

      Command: ttest DACC, by( year_cat) unequal
        _bs_1: r(t)

------------------------------------------------------------------------------
             |   Observed   Bootstrap                         Normal-based
             | coefficient  std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
       _bs_1 |    .714185   .9274841     0.77   0.441     -1.10365    2.532021
------------------------------------------------------------------------------

. use "C:\Users\user\Desktop\Daniel_bootstrap.dta", clear

. count if _bs_1>=0.7142
  535

. count if _bs_1<=-0.7142
  73

. di (535+73)/1000 *this is the bootstrap p-value you're interested in: bootstrap p-value=0.608*
.608

[Daniel Novak]

Hello Carlo,

thanks for the welcome and your quick and valuable response. I think that this is the answer to my question.
I will implement it to my data and look if it works.

[Carlo Lazzaro]

Daniel:
try:

. bootstrap r(t), reps(1000) strata(country_id firm_id)  saving(Daniel_bootstrap, every(1) double replace) nodots: ttest DACC, by( year_cat) unequal

[Daniel Novak]

Hello Carlo,

I think your approach is good and helps me to perform a t-test for the difference in means for the two periods.

Is it also possible to do the bootstrapping procedure with this steps?

1. Draw a randomly sample of X (e.g. 100) observations for each period (variable name: year) e.g.

sample 100, count by(year)

2. Perform regression for this sample and store the DACC measure

drop DACC //drop stored residuals from original regression
gen DACC = . 
reg lag_tacc inv_ta term2 lag_gppe mtbv lag_cfo,  const    
predict res 
replace DACC = res
drop res

3. Do steps 1 and 2 for X (e.g.1000) times
4. Perform t-test for differences in mean for whole bootstrap sample

I tried to implement this on my own, but I ran into problems, because I wanted to store the results of steps 1 and 2 and their replications in one dta file. Specifically the problems were, how to store the next drawn random sample in the same datafile and then create the the empty variable fields to store the res (=DACC).
Is it generally possible to do it this way or would you suggest something else?

[Carlo Lazzaro]

Daniel:
see -help forval-.
That said, what you're after is really challenging in terms of coding.
Are you sure that no other easier routes exist to accomplish what you're after?