Significance of coefficients after mata command bvpmvp

[Federico Tamagni]

Hi there. I am using the mata command bvpmvp by John Mullahy. I have 7 binary otucomes and I'm particularly interested in one covariate (call this Z) out of about 10 covariates. No problem with running the command and obtaining the "averaged" coefficients and standard errors for covariate Z and the other covariates. However, how can I test for significance of covariate Z on each of the 7 otucomes? Would it make sense to use the averaged standard errors on the covariate Z, to construct confidence intervals (at various confidence levels) via the usual benchmark-values of the standard normal distribution (e.g., 1.96 at 5% level, 2.575 at 1%, etc..)?

[John Mullahy]

Hello Federico. What you propose is probably the easiest approach. The appendix to my Stata Journal paper discusses possible drawbacks of using the reported std. errs..

If you wanted to be safe I suppose you could use bootstrapped std. errs. (just embed the bvpmvp function in a Mata loop). In fact it would be interesting to see how much the reported std. errs. differed from bootstrapped std. errs.

[Federico Tamagni]

Dear John, many thanks for your swift reply! I will surely consider your suggestion to compute bootstrapped s.e., as that sounds clearly to be the safest solution. Just one additional question about the "easiest solution", to be sure I made myself clear in the previous post. In a standard test of significance of a regression coefficient "beta", one builds the z-statistic as the ratio "beta/SE(beta)" and compares its value against critical values of the standard normal, at a given confidence level. However, after bvpmvp, one has an averaged beta and an averaged s.e. (in my case, averaged over the bivariate probits involving pairs of the 7 outcomes: y1-y2, y1-y3,y1-y4, y1-y5, y1-y6, y1-y7, y2-y3, y2-y4, y2-y5, etc...). Call the averaged beta and averaged s.e. as "AvgBeta" and "AvgSE", respectively. Question is: would it still hold that, in order to test significance of a given coefficient, one can build the "z-like" statistic "AvgBeta/AvgSE" and then compare the observed value against the critical values of the standard normal (as in the standard case where beta and the associated s.e. are not resulting from averaging)?

Thanks again!

[John Mullahy]

This is what I might do, Federico. That is, just treat the averages as if they are the relevant point estimates with the appropriate caution.

Of course an even simpler solution would be to estimate seven univariate probit models and just use their respective point estimates to compute Z-like statistics. If there's no specific interest in the jointness across outcomes then this seems the most straightforward approach.

As background, the reason I wrote the bvpmvp code was because I was specifically interested in estimating joint probabilities Pr(y1, ..., yM).

[Federico Tamagni]

Dear John, super useful. In fact, I'm also interested in joint probabilities. Many thanks again!!!