Testing for mediation using svy data

[rebecca eagleton]

Hi everyone,

I'm proposing a serial mediation model in Stata 16.0. The model uses complex sampling weights, so it requires the svy prefix. After searching online and on this forum, it appears that the best approach to test for significance of the indirect effect when using sampling weights is:

1. Use nlcom after running the SEM model

nlcom _b[mediator1:IV]*_b[mediator2:mediator1] *_b[DV:mediator2]

2. Obtain bootstrapped estimates

capture program drop bootmm
program bootmm, rclass
syntax [if] [in]
sureg (mediator1 IV)(mediator2 mediator1)(DV mediator2) `if' `in'
return scalar indtotal = _b[mediator1:IV]*_b[mediator2:mediator1] *_b[DV:mediator2]
end
bootstrap r(indtotal), bca reps(5000): bootmm

My model includes several control variables (modeled at each path, but eliminated here for clarity). My concern is that when I run the SEM model, the relationship between mediator2 and the DV is not significant.

Survey: Structural equation model Number of obs = 990
Number of strata = 6 Population size = 135,868,559
Number of PSUs = 990 Design df = 984

-------------------------------------------------------------------------------
| Linearized
| Coef. Std. Err. t P>|t| [95% Conf. Interval]
--------------+----------------------------------------------------------------
Structural |
mediator1 |
IV | .1898218 .0530235 3.58 0.000 .0857696 .2938739
_cons | .0778093 .0639975 1.22 0.224 -.047778 .2033966
------------+----------------------------------------------------------------
mediator2|
mediator1 | .097075 .0348129 2.79 0.005 .028759 .1653911
_cons | -.0149318 .0397547 -0.38 0.707 -.0929456 .063082
------------+----------------------------------------------------------------
DV |
mediator2 | -.1080093 .0944578 -1.14 0.253 -.2933711 .0773526
_cons | .0103036 .0602753 0.17 0.864 -.1079792 .1285865
------------+----------------------------------------------------------------


After running nlcom, the indirect effect is not significant (expected given the SEM results).


------------------------------------------------------------------------------
| Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
_nl_1 | -.0019903 .0019534 -1.02 0.308 -.0058189 .0018383
------------------------------------------------------------------------------


However, after obtaining the bootstrapped estimates, the parameters suggest that the indirect effect is now significant

Bootstrap results Number of obs = 990
Replications = 5,000

command: bootmm
_bs_1: r(indtotal)

------------------------------------------------------------------------------
| Observed Bootstrap Normal-based
| Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
_bs_1 | -.0036321 .0016421 -2.21 0.027 -.0068506 -.0004135
------------------------------------------------------------------------------


Am I using the right approach here? The significant indirect effect is unexpected given the lack of significance between mediator2 and the DV. I know that there is a medsem program, but it is unclear if this program can be used with sampling weights and for testing serial mediation.

Thank you for your help!

[Matthew Alexander]

Hi Rebecca,
You'll stand a greater chance of getting an informative response by making your post a little clearer. Code for the SEM model would help, for example.
In any case, it sounds to me that you are perhaps getting a little too hung up on p values and conventional significance levels. I note that despite a p-value of 0.3, the confidence interval is still fairly narrow. Why not present the SEM p-value, and SEM confidence limits and boostrapped SEM confidence limits,perhaps even at different confidence levels (say 95 and 99).
Without seeing code and output for your model, it's hard to say why p is above conventional significance levels. Often, you can find that simply doubling the no. of observations dramatically increases p (casting doubt on the substantive meaning of p). Or perhaps none of the paramters in your model are significant, in which case maybe you have omitted key predictors. Or maybe you did not mention that your model includes interactions terms. Without more information it's really hard to say what's going on here. At the very least, I would encourage you to take a broader view on significance, beyond conventional p value terms
All the very best,
Matt

[rebecca eagleton]

Hi Matt,

Thank you for your response. I have attached code for the SEM model (and reattached the model output) below. I'm trying to assess the significance of the indirect effect at a request of a manuscript reviewer. The relationships between (a) the IV and mediator 1, and (b) mediator 1 and mediator 2 are significant. However, the relationship between mediator 2 and the DV was not significant (which led me to question the significance of the indirect effect after bootstrapping). The model doesn't include interaction terms. I'm just concerned that I'm not actually running the analysis correctly. Using the medsem program produces the (non)significance I would expect, but as I mentioned above, I don't know of a way to use this program when testing serial mediation. Thank you again for your help!



svy linearized : sem (IV -> mediator1, ) (IV -> mediator2, ) (IV -> DV, ) (mediator1 -> mediator2, ) (mediator1-> DV, ) (mediator2 -> DV, ) (covariate1 -> mediator1, ) (covariate1-> mediator2, )
(covariate1-> DV, ) (covariate2-> mediator1, ) (covariate2-> mediator2,) (covariate2-> DV, ) (covariate3-> mediator1, ) (covariate3-> mediator2, ) (covariate3-> DV, ) (covariate4-> mediator1, ) (covariate4 -> mediator2, ) (covariate4 -> DV, ) (covariate5 -> mediator1, ) (covariate5 -> mediator2, ) (covariate5 -> DV, ) nocapslatent


Survey: Structural equation model Number of obs = 990
Number of strata = 6 Population size = 135,868,559
Number of PSUs = 990 Design df = 984

-------------------------------------------------------------------------------
| Linearized
| Coef. Std. Err. t P>|t| [95% Conf. Interval]
--------------+----------------------------------------------------------------
Structural |
Mediator1 |
IV | .1898218 .0530235 3.58 0.000 .0857696 .2938739
Covariate1 | .1430917 .0807475 1.77 0.077 -.0153655 .3015488
Covariate2 | .0053037 .0654473 0.08 0.935 -.1231286 .133736
Covariate3 | -.0576676 .0579433 -1.00 0.320 -.1713743 .0560392
Covariate4| .0731998 .0589586 1.24 0.215 -.0424991 .1888988
Covariate5 | .0329236 .0576997 0.57 0.568 -.080305 .1461522
_cons | .0778093 .0639975 1.22 0.224 -.047778 .2033966
------------+----------------------------------------------------------------
Mediator2|
Mediator1 | .097075 .0348129 2.79 0.005 .028759 .1653911
IV | .1057155 .0368578 2.87 0.004 .0333866 .1780444
Covariate1 | -.0542382 .0483536 -1.12 0.262 -.1491263 .0406499
Covariate2 | -.0328231 .0455944 -0.72 0.472 -.1222966 .0566503
Covariate3| .0068884 .0328247 0.21 0.834 -.0575261 .0713029
Covariate4 | .0276916 .0345896 0.80 0.424 -.0401862 .0955694
Covariate5 | -.0434932 .0371317 -1.17 0.242 -.1163595 .0293731
_cons | -.0149318 .0397547 -0.38 0.707 -.0929456 .063082
------------+----------------------------------------------------------------
DV |
Mediator1| .0256667 .0714991 0.36 0.720 -.1146415 .1659749
Mediator2| -.1080093 .0944578 -1.14 0.253 -.2933711 .0773526
IV | -.0781782 .066112 -1.18 0.237 -.2079148 .0515585
Covariate1 | -.0906719 .083675 -1.08 0.279 -.2548739 .07353
Covariate2| -.004884 .074011 -0.07 0.947 -.1501216 .1403535
Covariate3| .1946697 .0704039 2.77 0.006 .0565107 .3328286
Covariate4| .1195815 .0608905 1.96 0.050 .0000913 .2390717
Covariate5| .1649444 .0563374 2.93 0.003 .0543892 .2754997
_cons | .0103036 .0602753 0.17 0.864 -.1079792 .1285865

--------------+----------------------------------------------------------------
var(e. Mediator1)| .890133 .0658057 .7699276 1.029106
var(e. Mediator2)| .3386457 .0232696 .2959269 .3875312
var(e.DV) | .9777036 .2159705 .6337951 1.508223
--------------+----------------------------------------------------------------