Difference-in-Difference Estimation

[Luca Toni]

Hey,

I have some problem and hope someone can shed a light.

I'm trying to estimate the casual effect of UEFA Financial Fair Play (FFP) rules on the competitive balance of European football (soccer) leagues. Now, I don't know exactly how I'm going to measure competitiveness, but let's leave it since it's not my main issue.

I know that if I want to use DiD estimation, the treatment and control groups should have similar characteristics. This is why I cannot, for example, use the MLS league (USA) as a control group to the European leagues (USA teams don't participate in UEFA tournaments and are subject to different set of rules). So I thought about using the second leagues of these European leagues as control group. For example, using EFL Championship (England second league) as a control group, while the treatment will be the English Premier League, and so on for every league. The reason behind it is that football clubs in second leagues don't aspire to participate in UEFA competitions, hence they have no incentive to follow the FFP rules.

However, I question this choice (second league) as a control group. On the one hand, both the first and second leagues are subject to the same framework of rules because they belong to the same association in the country. On the other hand, teams in first and second leagues are not always similar in their budget, revenues, size, etc.

Another problem, by the way, is that some football teams, across the years, played both in the first and second leagues. I don't know how I can deal with that.

How can I approach it then using DiD estimation? Any suggestions?

I appreciate any help.

Thanks!

[Luca Toni]

Can someone help?

[George Ford]

As I see it, your main concern is parallel paths, not similar characteristics. If the MLS League provides a suitable counterfactual, then you can use it. The "similarity" is often important when we can't really assess parallel paths by more traditional means (looking at the data over time), and then largely by assumption (similar X means similar Y over time).

Having no incentive not to follow the FFP rules is not the same as not following them. I suppose you'd need some external evidence to suggests they do not, as a matter of course.

Budgets, etc, may be addressed by matching, but again PP is the issue, not necessarily similarity. I would think you'd have some time series data to look at to see if competitiveness follows a similar trend prior to the treatment.

The switching back and forth doesn't seem to be an issue, assuming the second leagues don't comply. But you could mark those for a richer DID model (it actually could be an advantage to your study). Really slick DID models have that sort of thing: A plant is proposed, sometimes built and sometimes not, which could detect selection bias (are plants proposed in growing labor markets, etc.).

[Luca Toni]

As I see it, your main concern is parallel paths, not similar characteristics. If the MLS League provides a suitable counterfactual, then you can use it. The "similarity" is often important when we can't really assess parallel paths by more traditional means (looking at the data over time), and then largely by assumption (similar X means similar Y over time).

Having no incentive not to follow the FFP rules is not the same as not following them. I suppose you'd need some external evidence to suggests they do not, as a matter of course.

Budgets, etc, may be addressed by matching, but again PP is the issue, not necessarily similarity. I would think you'd have some time series data to look at to see if competitiveness follows a similar trend prior to the treatment.

The switching back and forth doesn't seem to be an issue, assuming the second leagues don't comply. But you could mark those for a richer DID model (it actually could be an advantage to your study). Really slick DID models have that sort of thing: A plant is proposed, sometimes built and sometimes not, which could detect selection bias (are plants proposed in growing labor markets, etc.).

Thank you so much, George.
1) If identifying my control group is quite problematic, could I use Regression Discontinuity instead of DiD?
2) I actually don't know yet if I'm going to measure competitiveness on the club level or on the league level. The former makes it very complicated, I'm afraid. The latter is probably easier, but it will get me left with very few observations, and I'm not sure I can do casual inference with a very small sample in these methods (DiD/RD).
3) If my control assumption is somehow acceptable, I think that, in addition to second league teams, lower ranked teams in first leagues have no incentive as well to follow the FFP rules. Basically, I would say that the top 10 teams in each season might aspire to participate the UEFA competitions.
4) Finding financial data for earlier years in the sample (2012 and below), let alone for non-top-teams, is difficult. I can find aggregative financial data on the league level though, pretty much for all seasons.
5) I'm not entriely sure I understood what alternative DiD models can be used here. Would be helpful if you could elaborate a bit.

Thanks again. I really appreciate it.

[George Ford]

(1) possibly
(2) honestly, I'd start here. You need to figure out what you're dependent variable is going to be before you start thinking about how to model it.
(3) not sure how you look at lower level teams and get some sense of competitiveness, but this goes to (2)
(4) suggests league may be the best option, but how many do you have?
(5) you'd have a treatment*post indicator for teams that move in/out of the higher league, treating them differently than teams that never do (the pure control).

Do 2 first. I would think competitiveness would be measured by how irregular the ranking is, or maybe the ranking for the top N teams, or some such. Maybe build a simple simulation and play around to see what makes sense. You learn a lot about your problem when you're forced to construct an imaginary dataset and then build a model for it.

[Luca Toni]

(1) possibly
(2) honestly, I'd start here. You need to figure out what you're dependent variable is going to be before you start thinking about how to model it.
(3) not sure how you look at lower level teams and get some sense of competitiveness, but this goes to (2)
(4) suggests league may be the best option, but how many do you have?
(5) you'd have a treatment*post indicator for teams that move in/out of the higher league, treating them differently than teams that never do (the pure control).

Do 2 first. I would think competitiveness would be measured by how irregular the ranking is, or maybe the ranking for the top N teams, or some such. Maybe build a simple simulation and play around to see what makes sense. You learn a lot about your problem when you're forced to construct an imaginary dataset and then build a model for it.

Hey again.
before estimating the competitiveness on the league level, I decided to estimate the UEFA FFP rules on the overall transfer balance of the top European football clubs (that way I can test the break-even requirement of UEFA, which is maintaining a positive or a low negative balance of player transfers).
I collected data on each top club from 6 leagues in each season, including: transfer expenditure, overall balance (transfer income minus expenditure income), market value, and ratio of overall balance divided by market value which I think is the most important measure (will be the outcome variable) since the transfer market of 2005 is not the same of 2018 in terms of players valuations.
So I have 540 observations on the club level, between 2004 and 2023, and my cutoff for RDD is 2013.
However, when I run the simple command for RDD:

. rdrobust BalanceDeficittoMVratio Year, c(2013)
Mass points detected in the running variable.

Sharp RD estimates using local polynomial regression.

   Cutoff c = 2013 | Left of c  Right of c            Number of obs =        540
-------------------+----------------------            BW type       =      mserd
     Number of obs |       243         297            Kernel        = Triangular
Eff. Number of obs |        81         108            VCE method    =         NN
    Order est. (p) |         1           1
    Order bias (q) |         2           2
       BW est. (h) |     3.359       3.359
       BW bias (b) |     5.276       5.276
         rho (h/b) |     0.637       0.637
        Unique obs |         9          11

Outcome: BalanceDeficittoMVratio. Running variable: Year.
--------------------------------------------------------------------------------
            Method |   Coef.    Std. Err.    z     P>|z|    [95% Conf. Interval]
-------------------+------------------------------------------------------------
      Conventional | -.01857     .05967   -0.3112  0.756    -.13553      .098389
            Robust |     -          -     -0.1856  0.853    -.17216      .142379
--------------------------------------------------------------------------------
Estimates adjusted for mass points in the running variable.

I get this - "Mass points detected in the running variable" along with insignifcant coefficients.
What can I do now? I also suspects that since the FFP rules implemented gradually, starting at 2011 and fully implemented in 2013, it might influences the estimation. Question is how I can handle it.
Also, how can I find the optimal bandwidth? I tried to choose bandwidth but it gives me an error as well.

Here is an example of my dataset:

* Example generated by -dataex-. For more info, type help dataex
clear
input byte Place str16 Team byte(GP W D L GD) int(P Year) str22 League double(TransferExpenditure OverallBalanceDeficit MarketValue BalanceDeficittoMVratio)
 1 "Chelsea"         38 29  8  1  57  95 2004 "English Premier League"  162.4   159.1 331.48     .479968625558103
 2 "Arsenal"         38 25  8  5  51  83 2004 "English Premier League"  12.48    8.58    247  .034736842105263156
 3 "Manchester Utd"  38 22 11  5  32  77 2004 "English Premier League"  61.25   51.69 293.23   .17627800702520205
 5 "Liverpool"       38 17  7 14  11  58 2004 "English Premier League"   58.4   33.83 222.13   .15229820375455813
 8 "Manchester City" 38 13 13 12   8  52 2004 "English Premier League"    1.5   -6.44   94.3  -.06829268292682927
 1 "Chelsea"         38 29  4  5  50  91 2005 "English Premier League"   91.5    56.9 363.43   .15656384998486642
 2 "Manchester Utd"  38 25  8  5  38  83 2005 "English Premier League"   31.8    23.9 254.58   .09388011626993478
 3 "Liverpool"       38 25  7  6  32  82 2005 "English Premier League"  44.06   25.02 216.35   .11564594407210539
 4 "Arsenal"         38 20  7 11  37  67 2005 "English Premier League"     46      21 232.33   .09038867128653208
15 "Manchester City" 38 13  4 21  -5  43 2005 "English Premier League"  12.48  -20.03   91.9   -.2179542981501632
 1 "Manchester Utd"  38 28  5  5  56  89 2006 "English Premier League"   27.2     9.2 259.15   .03550067528458422
 2 "Chelsea"         38 24 11  3  40  83 2006 "English Premier League"  88.98   36.38 416.76   .08729244649198581
 3 "Liverpool"       38 20  8 10  30  68 2006 "English Premier League"     46   21.28 239.77   .08875172039871544
 4 "Arsenal"         38 19 11  8  28  68 2006 "English Premier League"     15    2.08 278.18   .00747717305341865
14 "Manchester City" 38 11  9 18 -15  42 2006 "English Premier League"    6.5     2.3     89  .025842696629213482
 1 "Manchester Utd"  38 27  6  5  58  87 2007 "English Premier League"  106.5    59.9 354.55   .16894655196728245
 2 "Chelsea"         38 25 10  3  39  85 2007 "English Premier League"     59   15.35 434.35   .03534016346264533
 3 "Arsenal"         38 24 11  3  43  83 2007 "English Premier League"  30.95  -25.78  262.4  -.09824695121951221
 4 "Liverpool"       38 21 13  4  39  76 2007 "English Premier League"   90.7   43.95 305.65   .14379191886144285
 9 "Manchester City" 38 15 10 13  -8  55 2007 "English Premier League"  77.95   67.39  150.1    .4489673550966023
 1 "Manchester Utd"  38 28  6  4  44  90 2008 "English Premier League"  45.25    37.8 425.95   .08874281018898932
 2 "Liverpool"       38 25 11  2  50  86 2008 "English Premier League"  71.45      25 342.85    .0729181857955374
 3 "Chelsea"         38 25  8  5  44  83 2008 "English Premier League"   30.5  -14.05 448.55  -.03132315237989076
 4 "Arsenal"         38 20 12  6  31  72 2008 "English Premier League"  40.15   14.35 311.45    .0460748113661904
10 "Manchester City" 38 15  5 18   8  50 2008 "English Premier League" 157.35  130.85 252.85    .5175004943642476
 1 "Chelsea"         38 27  5  6  71  86 2009 "English Premier League"     30    26.2 451.95  .057971014492753624
 2 "Manchester Utd"  38 27  4  7  58  85 2009 "English Premier League"   27.3  -77.17 384.88  -.20050405321139056
 3 "Arsenal"         38 23  6  9  42  75 2009 "English Premier League"     12   -35.7  314.3   -.1135857461024499
 5 "Manchester City" 38 18 13  7  28  67 2009 "English Premier League"  147.3   118.1  312.5               .37792
 7 "Liverpool"       38 18  9 11  26  63 2009 "English Premier League"   43.5   -5.05 331.55 -.015231488463278538
 1 "Manchester Utd"  38 23 11  4  41  80 2010 "English Premier League"   29.3   12.34    388  .031804123711340206
 2 "Chelsea"         38 21  8  9  36  71 2010 "English Premier League"  121.5     105  424.1   .24758311718934212
 3 "Manchester City" 38 21  8  9  27  71 2010 "English Premier League" 183.61  143.46 404.18   .35494086793013013
 4 "Arsenal"         38 19 11  8  29  68 2010 "English Premier League"     23    14.9    324  .045987654320987656
 6 "Liverpool"       38 17  7 14  15  58 2010 "English Premier League"  97.73   -3.78  339.2 -.011143867924528301
 1 "Manchester City" 38 28  5  5  64  89 2011 "English Premier League"  91.05   59.85  498.6   .12003610108303249
 2 "Manchester Utd"  38 28  5  5  56  89 2011 "English Premier League"   62.3   49.41  404.1   .12227171492204898
 3 "Arsenal"         38 21  7 10  25  70 2011 "English Premier League"  65.48  -12.82 362.43   -.0353723477637061
 6 "Chelsea"         38 18 10 10  19  64 2011 "English Premier League"  96.45   84.89  375.5   .22607190412782957
 8 "Liverpool"       38 14 10 14   7  52 2011 "English Premier League"  69.33   47.33  236.9    .1997889404812157
 1 "Manchester Utd"  38 28  5  5  43  89 2012 "English Premier League"  76.45    66.8  457.5   .14601092896174864
 2 "Manchester City" 38 23  9  6  32  78 2012 "English Premier League"  61.95   17.65  502.1  .035152360087631944
 3 "Chelsea"         38 22  9  7  36  75 2012 "English Premier League"  109.7   84.25 454.25   .18547055586130984
 4 "Arsenal"         38 21 10  7  35  73 2012 "English Premier League"     56   -9.85  296.9 -.033176153587066354
 7 "Liverpool"       38 16 13  9  28  61 2012 "English Premier League"   70.6   60.15  276.1    .2178558493299529
 1 "Manchester City" 38 27  5  6  65  86 2013 "English Premier League"  115.5   104.2  455.8   .22860903905221588
 2 "Liverpool"       38 26  6  6  51  84 2013 "English Premier League"   58.1    25.6 315.75   .08107680126682502
 3 "Chelsea"         38 25  7  6  44  82 2013 "English Premier League" 130.35   52.42  584.2   .08972954467648066
 4 "Arsenal"         38 24  7  7  27  79 2013 "English Premier League"  49.25    37.1 385.75   .09617627997407648
 7 "Manchester Utd"  38 19  7 12  21  64 2013 "English Premier League"  77.13   75.33 402.55   .18713203328779032
 1 "Chelsea"         38 26  9  3  41  87 2014 "English Premier League"  137.7   -5.11 557.25 -.009170031404217138
 2 "Manchester City" 38 24  7  7  45  79 2014 "English Premier League"  102.8    72.5 448.25   .16174010039040715
 3 "Arsenal"         38 22  9  7  35  75 2014 "English Premier League" 118.98   91.18 414.75   .21984327908378543
 4 "Manchester Utd"  38 20 10  8  25  70 2014 "English Premier League" 195.35  148.65  505.4    .2941234665611397
 6 "Liverpool"       38 18  8 12   4  62 2014 "English Premier League" 151.43   52.16 324.85   .16056641526858548
 2 "Arsenal"         38 20 11  7  29  71 2015 "English Premier League"   26.5      24 457.85  .052418914491645735
 4 "Manchester City" 38 19  9 10  30  66 2015 "English Premier League" 208.47  141.03 499.35   .28242715530189244
 5 "Manchester Utd"  38 19  9 10  14  66 2015 "English Premier League"    156   55.33 439.85   .12579288393770602
 8 "Liverpool"       38 16 12 10  13  60 2015 "English Premier League"  126.5   35.95 375.25   .09580279813457696
10 "Chelsea"         38 12 14 12   6  50 2015 "English Premier League"   96.5    9.01  553.5  .016278229448961155
 1 "Chelsea"         38 30  3  5  52  93 2016 "English Premier League"  132.8    23.9  686.7   .03480413572156691
 3 "Manchester City" 38 23  9  6  41  78 2016 "English Premier League" 216.25   180.9  576.5   .31379011274934954
 4 "Liverpool"       38 22 10  6  36  76 2016 "English Premier League"   79.9   -5.48 425.65 -.012874427346411373
 5 "Arsenal"         38 23  6  9  33  75 2016 "English Premier League"    113  102.65    542   .18939114391143913
 6 "Manchester Utd"  38 18 15  5  25  69 2016 "English Premier League"    185  137.75 531.45   .25919653777401447
 1 "Manchester City" 38 32  4  2  79 100 2017 "English Premier League"  317.5  226.15   1010   .22391089108910892
 2 "Manchester Utd"  38 25  6  7  40  81 2017 "English Premier League"  198.4   152.9  849.5    .1799882283696292
 4 "Liverpool"       38 21 12  5  46  75 2017 "English Premier League" 173.65  -10.85  857.5 -.012653061224489795
 5 "Chelsea"         38 21  7 10  24  70 2017 "English Premier League"  260.5    65.9 884.25   .07452643483177834
 6 "Arsenal"         38 19  6 13  23  63 2017 "English Premier League" 152.85   -9.55    694 -.013760806916426513
 1 "Manchester City" 38 32  2  4  72  98 2018 "English Premier League"  78.59   20.99   1200  .017491666666666666
 2 "Liverpool"       38 30  7  1  67  97 2018 "English Premier League"  182.2  140.88   1170    .1204102564102564
 3 "Chelsea"         38 21  9  8  24  72 2018 "English Premier League"  208.8  125.55   1170    .1073076923076923
 5 "Arsenal"         38 21  7 10  22  70 2018 "English Premier League"  80.15   71.05 659.05   .10780669144981413
 6 "Manchester Utd"  38 19  9 10  11  66 2018 "English Premier League"   82.7   52.15  797.6   .06538365095285857
 1 "Liverpool"       38 32  3  3  52  99 2019 "English Premier League"   10.4   -37.6   1000               -.0376
 2 "Manchester City" 38 26  3  9  67  81 2019 "English Premier League" 169.82   98.82   1050    .0941142857142857
 3 "Manchester Utd"  38 18 12  8  30  66 2019 "English Premier League"  236.8  155.62 670.45   .23211276008650905
 4 "Chelsea"         38 20  6 12  15  66 2019 "English Premier League"     45 -118.07 705.85  -.16727350003541827
 8 "Arsenal"         38 14 14 10   8  56 2019 "English Premier League"  160.8  107.15 607.65    .1763350613017362
 1 "Manchester City" 38 27  5  6  51  86 2020 "English Premier League"  173.4   109.4   1040    .1051923076923077
 2 "Manchester Utd"  38 21 11  6  29  74 2020 "English Premier League"   83.8    64.3 770.05   .08350107135900266
 3 "Liverpool"       38 20  9  9  26  69 2020 "English Premier League"  84.05   66.85 969.95    .0689210784061034
 4 "Chelsea"         38 19 10  9  22  67 2020 "English Premier League"  247.2  190.05  889.2    .2137314439946019
 8 "Arsenal"         38 18  7 13  16  61 2020 "English Premier League"     86   66.85 619.25   .10795316915623737
 1 "Manchester City" 38 29  6  3  73  93 2021 "English Premier League"  138.9    45.1   1000                .0451
 2 "Liverpool"       38 28  8  2  68  92 2021 "English Premier League"     90   60.45  918.9   .06578517793013386
 3 "Chelsea"         38 21 11  6  43  74 2021 "English Premier League"    118  -36.69  914.5  -.04012028430836522
 5 "Arsenal"         38 22  3 13  13  69 2021 "English Premier League"  167.4     136 613.05   .22184161161406085
 6 "Manchester Utd"  38 16 10 12   0  58 2021 "English Premier League"    142   110.9 769.15   .14418513943964117
 1 "Manchester City" 38 28  5  5  61  89 2022 "English Premier League"    155   -7.17   1150 -.006234782608695652
 2 "Arsenal"         38 26  6  6  45  84 2022 "English Premier League"  186.4   162.6   1000                .1626
 3 "Manchester Utd"  38 23  6  9  15  75 2022 "English Premier League" 243.28  219.63    848   .25899764150943394
 5 "Liverpool"       38 19 10  9  28  67 2022 "English Premier League"  142.3    61.6 811.85   .07587608548377163
12 "Chelsea"         38 11 11 16  -9  44 2022 "English Premier League" 630.35  562.49 994.95    .5653449922106638
 1 "Manchester City" 38 28  7  3  62  91 2023 "English Premier League"  259.6   133.8   1270   .10535433070866143
 2 "Arsenal"         38 28  5  5  62  89 2023 "English Premier League" 234.94  165.74   1140    .1453859649122807
 3 "Liverpool"       38 24 10  4  45  82 2023 "English Premier League"    172   111.3  920.4   .12092568448500651
 6 "Chelsea"         38 18  9 11  14  63 2023 "English Premier League"  471.8   202.3    912    .2218201754385965
 8 "Manchester Utd"  38 18  6 14  -1  60 2023 "English Premier League"  202.3  143.96 715.25    .2012722824187347
end

Thank you.

[Luca Toni]

Still looking for help.

[Frode Andre]

The code works fine with your example dataset. I am using Stata 18.

[Luca Toni]

The code works fine with your example dataset. I am using Stata 18.

Hi. Thanks, but my question were:
1) when I get "Mass points detected in the running variable" along with insignificant coefficients, how can I deal with that?
2) I also suspect that since the FFP rules implemented gradually, starting in 2011 and fully implemented in 2013, it might influences the estimation. Question is how I can handle it?
3) how can I find the optimal bandwidth? I tried to choose bandwidth but it gives me an error as well.

[George Ford]

could add masspoints(adjust), but that's the default I think. You'll see at the bottom it says adjusted for mass points.

The insignificance indicates that the null is not rejected. It is not a fault in the code.

[Luca Toni]

could add masspoints(adjust), but that's the default I think. You'll see at the bottom it says adjusted for mass points.

The insignificance indicates that the null is not rejected. It is not a fault in the code.

Thank you.
1) Yes, it appears to be the default.
2) Is it likely that the insignificance stems from the fact that I didn't control for covariates? or maybe because I didn't use the optimal bandwidth? I guess the low number of observations plays a role too.

[George Ford]

Or, maybe there's no change.

Have you stared at the data (scatters, etc..) to see if there's a change?

[Luca Toni]

Or, maybe there's no change.

Have you stared at the data (scatters, etc..) to see if there's a change?

Yes. When I choose the cutoff to be 2011 instead of 2013, I get much less insignificant coefficient (pv is 0.2) and there is a sharp change (on graph) in this year. I think I should define my covariates and then run it again. I'm not surprised that in 2011 there is a change (although still insignificant) since the FFP rules were gradually implemented starting in 2011. Maybe I should use fuzzy RD instead of sharp RD?