ivregress and complier average causa effect (CACE) analysis

[Francesca Pesola]

Dear all,

I need to run a CACE analysis and I found some online references suggesting this can be done in Stata using the -ivregress- command. I am analysing data from a RCT and I found an effect of intervention arm on the main outcome as part of my ITT analysis; however, it has been suggested I should now use a causal model approach with instrumental variables to be able to compare the compliers in the intervention group with the 'compliers' in the control group, had they been offered the intervention.

Compliance is defined as attending all sessions. I have identified predictors of compliance in the study arm and used these predictors to identify the 'compliers' in the control group. Now I can use -ivregress- as follows:

xi: ivregress 2sls fSFpro bSFpro i.centre (comply=arm)

BUT the results are unreasonable as the coefficient of 'comply' on the outcome is very large (my scale only ranges from 0-33 on a continuous) - attached doc - the b = 537 and 95%CI: -4934 to 6008.

I would welcome any suggestion on what could be going on and wrong and ways to rectify it.

Many thanks in advance,
Francesca


table.docx

[Alexander Wuttke]

Have you found a solution to your problem?

[Francesca Pesola]

Hey Alexander,

I never did. Have you got a solution?

[Roman Mostazir]

Francesca, the research in CACE is a growing interest. When you have small sample size, the IV approach may return volatile results if compliance rate is low (look at this paper). Since IV regression is running on two-least square, it is running a complete case estimation which makes the model more volatile as data are dropped out if missing. In these case, ML/Baysean approach with pretreatment covariate may achieve better performace. The result for IV you presented, how much the coefficient varies with ITT ? Usually CACE estimates are inflated compared to ITT estimates. Here is an example with hypothetical data but with 10000 sample:

//We hypothesize that intervention reduced BMI for Treatment group:

//Fake data generation:

set obs 10000

gen id =_n
gen rand = runiform()>.5 //randomised arm
lab define rand 0 "Control" 1 "Treat"
lab val rand rand

gen bmi0 = float((34-25)*runiform()+25) //Baseline BMI

gen bmi1 = float((34-25)*runiform()+25) if rand==0 //Post treatment BMI for Control

replace bmi1 = float((27-23)*runiform()+25) if rand==1 //Post treatment BMI for Treatment group.

//ITT analysis:

reg bmi1 bmi0 rand //Baseline adjusted treatment effect: ITT


      Source |       SS           df       MS      Number of obs   =    10,000
-------------+----------------------------------   F(2, 9997)      =   1857.65
       Model |  14964.9011         2  7482.45054   Prob > F        =    0.0000
    Residual |  40267.0708     9,997  4.02791545   R-squared       =    0.2709
-------------+----------------------------------   Adj R-squared   =    0.2708
       Total |  55231.9719     9,999  5.52374956   Root MSE        =     2.007

------------------------------------------------------------------------------
        bmi1 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        bmi0 |   .0014873   .0077126     0.19   0.847     -.013631    .0166055
        rand |  -2.446603   .0401395   -60.95   0.000    -2.525284   -2.367921
       _cons |   29.43365   .2295339   128.23   0.000     28.98372    29.88359
------------------------------------------------------------------------------

//ITT suggest treatment group had BMI lowered by -2.5 units

//Lets see the complier effect. We assume compliers had greater treatment effect (for demo purpose)

gen comply = cond(bmi1<27 & rand==1,1,0)

//IV regression:

ivregress 2sls bmi1 bmi0 (comply = rand)
Instrumental variables (2SLS) regression          Number of obs   =     10,000
                                                  Wald chi2(2)    =    3206.28
                                                  Prob > chi2     =     0.0000
                                                  R-squared       =     0.1550
                                                  Root MSE        =     2.1604

------------------------------------------------------------------------------
        bmi1 |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      comply |  -4.994759   .0882103   -56.62   0.000    -5.167648    -4.82187
        bmi0 |   .0063698   .0083026     0.77   0.443     -.009903    .0226425
       _cons |   29.28946   .2468681   118.64   0.000     28.80561    29.77332
------------------------------------------------------------------------------

// Compliers effect is -5 unit and greater than ITT.

// We can estimate the same compliance effect using Structural Equation Model


sem (comply <- rand) (bmi1 <-comply bmi0), cov(e.comply*e.bmi1)

-------------------------------------------------------------------------------------
                    |                 OIM
                    |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
--------------------+----------------------------------------------------------------
Structural          |
  comply <-         |
               rand |   .4898244   .0070781    69.20   0.000     .4759516    .5036972
              _cons |   6.07e-14    .005011     0.00   1.000    -.0098213    .0098213
  ------------------+----------------------------------------------------------------
  bmi1 <-           |
             comply |  -4.994818    .088213   -56.62   0.000    -5.167712   -4.821923
               bmi0 |   .0034279   .0072235     0.47   0.635    -.0107299    .0175858
              _cons |   29.37634   .2155065   136.31   0.000     28.95396    29.79873
--------------------+----------------------------------------------------------------
       var(e.comply)|   .1252481   .0017713                      .1218241    .1287683
         var(e.bmi1)|   4.667477   .0936998                      4.487395    4.854786
--------------------+----------------------------------------------------------------
cov(e.comply,e.bmi1)|   .3769366    .013955    27.01   0.000     .3495854    .4042879
-------------------------------------------------------------------------------------

This is a loose demo for CACE applicable only if you have randomised arms restricted to switch between arms. I recommend this paper for a proper understanding of the whole idea of Principal Stratification of which CACE is a special case.

PS: For your model, did you try dropping the exogenous variables? If you have a proper randomzation, probably you don't need them in your model, or better to assess how much they are contributing towards the IV model. I would go with a simpler model first.

[Francesca Pesola]

Hi Roman,

Thank you for the example. I will also check out the paper!

Francesca