How to perform an equivalence test (based on a t-test) for regression coefficients?

[Olaf Hotte]

Hi,

How do I perform an equivalence test (based on a t-test) for regression coefficients in STATA?
I do know how to perform one- or two-sided t-test, but I couldn't find a command to perform a equivalence test. Here is a quick example:

. sysuse auto, clear
(1978 Automobile Data)

. regress price mpg weight

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(2, 71)        =     14.74
       Model |   186321280         2  93160639.9   Prob > F        =    0.0000
    Residual |   448744116        71  6320339.67   R-squared       =    0.2934
-------------+----------------------------------   Adj R-squared   =    0.2735
       Total |   635065396        73  8699525.97   Root MSE        =      2514

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |  -49.51222   86.15604    -0.57   0.567    -221.3025     122.278
      weight |   1.746559   .6413538     2.72   0.008      .467736    3.025382
       _cons |   1946.069    3597.05     0.54   0.590    -5226.245    9118.382
------------------------------------------------------------------------------

two-sided t-test:

e.g. mpg = 0

. test _b[mpg]=0

 ( 1)  mpg = 0

       F(  1,    71) =    0.33
            Prob > F =    0.5673

one-sided t-test:

e.g. mpg >= 0

. local sign_mpg = sign(_b[mpg])

. display "Ho: coef >= 0  p-value = " 1-ttail(r(df_r),`sign_mpg'*sqrt(r(F)))
Ho: coef >= 0  p-value = .28366186

equivalence test:

e.g. |mpg| >= 100
(mpg >= 100 and mpg <= -100)

How do I perfrom this test?

[Joseph Coveney]

equivalence test:

e.g. |mpg| >= 100
(mpg >= 100 and mpg <= -100)

How do I perform this test?

Step 1: After fitting the model that you show, type

regress , level(90)

Step 2: Look at the regression output's table of coefficients (it will be very similar to the one that you show but with a slightly different text in the header for the two rightmost columns). Scan to the two rightmost columns' header that says "[90% conf. interval]"

Step 3: Scan down the two columns of confidence bounds until you're at the row for mpg (it's the first row in your example)

Final step: You stated the null hypothesis for your equivalence test, and so the alternative hypothesis would be cast at as "|mpg| < 100, i.e., (mpg < 100 and mpg > -100)". Therefore, if the lower confidence bound is greater (more positive) than -100 and the upper confidence bound is less than 100, then you're entitled to reject your stated null hypothesis and to declare in favor of the alternative hypothesis of so-called equivalence at the 5% level of type I error rate for the range of ±100 mpg, exclusive.

[Olaf Hotte]

Ok, great. Thank you. This approach is working, but I was hopping for a more analytical way.

I came up with this:

. di 1-ttail(r(df_r),(abs(_b[mpg])-100)/_se[mpg])
.27986559

This should be correct as well, right?

[Olaf Hotte]

Here is a more detailled comparision and little correction:

. di (1-ttail(e(df_r),(abs(_b[mpg])-100)/_se[mpg]))*2
.55973118

. di 1-.55973118
.44026882

. regress , level(44.03)

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(2, 71)        =     14.74
       Model |   186321280         2  93160639.9   Prob > F        =    0.0000
    Residual |   448744116        71  6320339.67   R-squared       =    0.2934
-------------+----------------------------------   Adj R-squared   =    0.2735
       Total |   635065396        73  8699525.97   Root MSE        =      2514

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|  [44.03% Conf. Interval]
-------------+----------------------------------------------------------------
         mpg |  -49.51222   86.15604    -0.57   0.567     -100.004    .9795777
      weight |   1.746559   .6413538     2.72   0.008     1.370693    2.122425
       _cons |   1946.069    3597.05     0.54   0.590    -161.9844    4054.122
------------------------------------------------------------------------------

Both metods result in alpha = 0.56

This should be correct, right?

[Felix Bittmann]

Maybe this package is helpful:

findit tost
tostregress tests for the equivalence of each regression coefficient and
    zero within separate symmetric equivalence intervals defined by eqvtype
    and eqvlevel for using a two one-sided t tests approach (Schuirmann,
    1987).  Typically ('positivist') null hypotheses are framed from an
    assumption of a lack of difference between two quantities, and reject
    this assumption only with sufficient evidence.  When performing tests for
    equivalence, one frames a ('negativist') null hypothesis with the
    assumption that two quantities are different by at least as much as an
    equivalence interval defined by some chosen level of tolerance.

[Joseph Coveney]

Both metods result in alpha = 0.56

This should be correct, right?

If you go that route, then you'll want to cover both sides.

.ÿ
.ÿversionÿ17.0

.ÿ
.ÿclearÿ*

.ÿ
.ÿquietlyÿsysuseÿauto

.ÿ
.ÿ*
.ÿ*ÿBeginÿhere
.ÿ*
.ÿquietlyÿregressÿpriceÿc.(mpgÿweight)

.ÿ
.ÿquietlyÿtestÿ_b[mpg]ÿ=ÿ-100

.ÿtempnameÿleft

.ÿscalarÿdefineÿ`left'ÿ=ÿr(p)

.ÿ
.ÿquietlyÿtestÿ_b[mpg]ÿ=ÿ100

.ÿdisplayÿinÿsmclÿasÿtextÿ_newline(1)ÿ"TestÿofÿH0:ÿ-100ÿ<=ÿmuÿ<=ÿ100:ÿpÿ=ÿ"ÿ///
>ÿÿÿÿÿÿÿÿÿasÿresultÿ%6.4fÿmax(`left',ÿr(p))

TestÿofÿH0:ÿ-100ÿ<=ÿmuÿ<=ÿ100:ÿpÿ=ÿ0.5597

.ÿ
.ÿexit

endÿofÿdo-file


.

If you had got, say, p = 0.04 for +100 and p = 0.09 for -100, what would you have concluded about equivalence?

I'd go with the confidence interval approach, myself. You can judge immediately by inspection (it's a faster, more direct and more readily interpretable answer to the research question), and it helps avoid explicit reference to p-values that might give rise to philosophical objection by a referee.