Can I test differences between two regression coefficients in linear mixed regression?

[Marco Lazzeretti]

Hi, to everyone! I have this problem and I don't know to resolve it.
I have a DATA PANEL database and I would estimate the effects of time (tempomesi) and treatment (TRATTATO, the same during the study) on my dependent variable (ILD). For this reason, I choose to use mixed effects models, with varying-intercept and varying-slope, using this command:

xtmixed ILD tempomesi TRATTATO || subject: tempomesi TRATTATO, mle nolog covariance(unstructure)

Treatment variable does not significative. But, I suppose, that the effect of time is different in two groups and therefore I try to estimate two different models, by treatment variable, in this way:

bysort TRATTATO: xtmixed ILD tempomesi || subject: tempomesi, mle nolog covariance(unstructure)

I have obtained:

----------------------------------------------------------------------------------------------------------------------------------
-> TRATTATO = 0

Mixed-effects ML regression Number of obs = 62
Group variable: subject Number of groups = 27

Obs per group: min = 2
avg = 2.3
max = 6


Wald chi2(1) = 21.66
Log likelihood = -245.84494 Prob > chi2 = 0.0000

------------------------------------------------------------------------------
ILD | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
tempomesi | .4656706 .100053 4.65 0.000 .2695703 .6617709
_cons | 20.01422 3.260442 6.14 0.000 13.62387 26.40457
------------------------------------------------------------------------------

------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
subject: Unstructured |
sd(tempom~i) | .2559976 .1101921 .1101141 .5951532
sd(_cons) | 15.76703 2.524862 11.51973 21.5803
corr(tempom~i,_cons) | 1 .0000684 -1 1
-----------------------------+------------------------------------------------
sd(Residual) | 6.931615 .8303102 5.48116 8.765899
------------------------------------------------------------------------------
LR test vs. linear regression: chi2(3) = 48.18 Prob > chi2 = 0.0000

Note: LR test is conservative and provided only for reference.

----------------------------------------------------------------------------------------------------------------------------------
-> TRATTATO = 1

Mixed-effects ML regression Number of obs = 55
Group variable: subject Number of groups = 25

Obs per group: min = 2
avg = 2.2
max = 4


Wald chi2(1) = 3.09
Log likelihood = -219.53361 Prob > chi2 = 0.0787

------------------------------------------------------------------------------
ILD | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
tempomesi | .2239693 .1273935 1.76 0.079 -.0257174 .4736559
_cons | 18.23112 3.346141 5.45 0.000 11.6728 24.78943
------------------------------------------------------------------------------

------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
subject: Unstructured |
sd(tempom~i) | .0750234 .0589198 .0160955 .3496951
sd(_cons) | 14.96999 2.525032 10.7559 20.83513
corr(tempom~i,_cons) | .9999789 .0014601 -1 1
-----------------------------+------------------------------------------------
sd(Residual) | 7.842885 1.110046 5.942932 10.35025
------------------------------------------------------------------------------
LR test vs. linear regression: chi2(3) = 30.35 Prob > chi2 = 0.0000

Note: LR test is conservative and provided only for reference.




I will be interested to test differences between regression coefficients (tempomesi|TRATTATO=0)=0.4657 and (tempomesi|TRATTATO=1)=0.2240. Could i do this in STATA?

Thanks

Marco

[Rich Goldstein]

your post is very hard to read; please read the FAQ on how to post using CODE blocks to make it easier to read

it appears that you are estimating two models; why not use one model with an interaction between TRATTATO and tempomesi?

[Marco Lazzeretti]

Ok, thank you for information. I've tried to use those commands:

generate trat_tempomesi=TRATTATO*tempomesi
xtmixed ILD tempomesi TRATTATO trat_tempomesi || subject: tempomesi, mle nolog covariance(unstructured)

but I've obtained coefficients a lit bit different:

tempomesi 0.4540
TRATTATO -2.0295
trat_tempomesi -0.1947
_cons 20.0924

and only tempomesi is significative.

[Carlo Lazzaro]

Marco:
the results you posted are incomplete and hard to read due to formatting issues. Please, use CODE blocks, as recommended by Rich.
That said, why creating interaction by hand when -fvvarlist- can do it for you and link your results to two wonderful postestimation commands such as -margins. and -marginsplot-?
Eventually, the presence/absence of statistical significant can depend on many factors.
For instance, assuming that -TRATTATO- (that means treated for those not familiar with the Italian language) is a categorical variable and -tempomesi- (that means time measured in months for those...) is a continuous variable, I would re-run your model with the folowing code for interaction:

i.TRATTATO##c.tempomesi

[Marco Lazzeretti]

TRATTATO (0/1) is a time-invariant variable. Can I treat as like tempomesi?

[Carlo Lazzaro]

Marco:
I do not think so.
You should keep it categorical.

[Marco Lazzeretti]

There are the output of two models in this file. But the results are a little bit different

[Carlo Lazzaro]

Marco:
please read the FAQ about posting attachments (especially spreadsheet, which are usually ignored): you fill find out that there are more fruitful ways to share what you have done with other listers. Thanks.

[Marco Lazzeretti]

bysort TRATTATO: xtmixed ILD tempomesi || subject: tempomesi, mle nolog covariance(unstructure)

generate trat_tempomesi=TRATTATO*tempomesi
xtmixed ILD tempomesi TRATTATO trat_tempomesi || subject: tempomesi, mle nolog covariance(unstructured)

This two codes give different results

[Marco Lazzeretti]

----------------------------------------------------------------------------------------------------------------------------------
-> TRATTATO = 0

Mixed-effects ML regression                     Number of obs      =        62
Group variable: subject                         Number of groups   =        27

                                                Obs per group: min =         2
                                                               avg =       2.3
                                                               max =         6


                                                Wald chi2(1)       =     21.66
Log likelihood = -245.84494                     Prob > chi2        =    0.0000

------------------------------------------------------------------------------
         ILD |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   tempomesi |   .4656686   .1000521     4.65   0.000     .2695701    .6617671
       _cons |   20.01424   3.260427     6.14   0.000     13.62392    26.40456
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
subject: Unstructured        |
                sd(tempom~i) |   .2559839   .1101932      .1101024    .5951533
                   sd(_cons) |   15.76693   2.524848      11.51965    21.58017
        corr(tempom~i,_cons) |          1   .0000603            -1           1
-----------------------------+------------------------------------------------
                sd(Residual) |   6.931679   .8303264      5.481197    8.766001
------------------------------------------------------------------------------
LR test vs. linear regression:       chi2(3) =    48.18   Prob > chi2 = 0.0000

Note: LR test is conservative and provided only for reference.

----------------------------------------------------------------------------------------------------------------------------------
-> TRATTATO = 1

Mixed-effects ML regression                     Number of obs      =        55
Group variable: subject                         Number of groups   =        25

                                                Obs per group: min =         2
                                                               avg =       2.2
                                                               max =         4


                                                Wald chi2(1)       =      3.09
Log likelihood = -219.53361                     Prob > chi2        =    0.0787

------------------------------------------------------------------------------
         ILD |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   tempomesi |   .2239693   .1273935     1.76   0.079    -.0257173    .4736559
       _cons |   18.23112   3.346141     5.45   0.000      11.6728    24.78943
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
subject: Unstructured        |
                sd(tempom~i) |   .0750236   .0580362      .0164714    .3417153
                   sd(_cons) |   14.96999   2.528887      10.75047    20.84565
        corr(tempom~i,_cons) |   .9999785   .0013941            -1           1
-----------------------------+------------------------------------------------
                sd(Residual) |   7.842884   1.116699      5.933058    10.36748
------------------------------------------------------------------------------
LR test vs. linear regression:       chi2(3) =    30.35   Prob > chi2 = 0.0000

[Carlo Lazzaro]

Marco:
you ran two different models: hence, no wonder that results are different.
Just to investigate whether interaction in the fixed portion of your model makes sense, have you tried:

xtmixed ILD c.tempomesi##i.TRATTATO trat_tempomesi || subject: tempomesi, mle nolog covariance(unstructured)

Moreover: have you checked whether random coefficient model outperforms the random intercept one?

[Marco Lazzeretti]

. xtmixed ILD tempomesi TRATTATO trat_tempomesi || subject: tempomesi, mle nolog covariance(unstructured)

Mixed-effects ML regression                     Number of obs      =       117
Group variable: subject                         Number of groups   =        52

                                                Obs per group: min =         2
                                                               avg =       2.3
                                                               max =         6


                                                Wald chi2(3)       =     25.71
Log likelihood = -466.17812                     Prob > chi2        =    0.0000

------------------------------------------------------------------------------
         ILD |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   tempomesi |   .4539967   .0977982     4.64   0.000     .2623158    .6456776
    TRATTATO |  -2.029481   4.656899    -0.44   0.663    -11.15684    7.097875
trat_tempo~i |  -.1946891   .1604875    -1.21   0.225    -.5092388    .1198606
       _cons |   20.09244   3.209618     6.26   0.000     13.80171    26.38318
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
subject: Unstructured        |
                sd(tempom~i) |   .1892336   .0866268      .0771503    .4641504
                   sd(_cons) |   15.31787   1.813277      12.14607    19.31793
        corr(tempom~i,_cons) |   .9999995   .0004222            -1           1
-----------------------------+------------------------------------------------
                sd(Residual) |   7.399547   .6513739      6.226942    8.792967
------------------------------------------------------------------------------
LR test vs. linear regression:       chi2(3) =    77.37   Prob > chi2 = 0.0000

Note: LR test is conservative and provided only for reference.

[Marco Lazzeretti]

This is the output with your code

note: 1.TRATTATO#c.tempomesi omitted because of collinearity

Mixed-effects ML regression                     Number of obs      =       117
Group variable: subject                         Number of groups   =        52

                                                Obs per group: min =         2
                                                               avg =       2.3
                                                               max =         6


                                                Wald chi2(3)       =     25.71
Log likelihood = -466.17812                     Prob > chi2        =    0.0000

------------------------------------------------------------------------------
         ILD |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
   tempomesi |   .4539967   .0977982     4.64   0.000     .2623158    .6456776
  1.TRATTATO |  -2.029481   4.656899    -0.44   0.663    -11.15684    7.097875
             |
    TRATTATO#|
 c.tempomesi |
          1  |  (omitted)
             |
trat_tempo~i |  -.1946891   .1604875    -1.21   0.225    -.5092388    .1198606
       _cons |   20.09244   3.209618     6.26   0.000     13.80171    26.38318
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
subject: Unstructured        |
                sd(tempom~i) |   .1892336   .0866268      .0771503    .4641504
                   sd(_cons) |   15.31787   1.813277      12.14607    19.31793
        corr(tempom~i,_cons) |   .9999995   .0004222            -1           1
-----------------------------+------------------------------------------------
                sd(Residual) |   7.399547   .6513739      6.226942    8.792967
------------------------------------------------------------------------------
LR test vs. linear regression:       chi2(3) =    77.37   Prob > chi2 = 0.0000

[Carlo Lazzaro]

Marco:
as per your resulsts, there's no evidence of an interaction effect between -tempomesi- and -TRATTATO-.
However, you did not check whether random coefficient model outperforms the random intercept one, but whether random coefficient model outperforms OLS.
See -lrtest- under -mixed postestimation- entry in Stata .pdf manual.

[Marco Lazzeretti]

I've tried to compare the model with only-varying intercept (ri) versus varying-all (rc) and the results are those:

quietly xtmixed ILD tempomesi || subject:, mle nolog
estimates store ri

quietly xtmixed ILD tempomesi || subject: tempomesi, mle nolog covariance(unstructure)
estimates store rc

lrtest ri rc

. lrtest ri rc

Likelihood-ratio test                                  LR chi2(2)  =      5.72
(Assumption: ri nested in rc)                          Prob > chi2 =    0.0572

Note: The reported degrees of freedom assumes the null hypothesis is not on the boundary of the parameter space.  If this is not
      true, then the reported test is conservative.

But with treatment values (0/1) not