Statistical significance of the difference in the two marginal effects estimates from two subsamples with same model

[Amruta Bagwe]

How to check if the difference in the marginal effects of the treatment variable in two subsamples estimated using the same model is statistically significant?
I want to check if the marginal effects in a probit model that estimates the impact of a policy shock on the probability of employment are different for men and women. I first estimate the two probit models for men and women and calculate their Marginal effects. However, I want to know whether the difference in the estimates of the marginal effects of the policy shock for men and women is statistically significant.
My codes are as follows:
Model 1 Men: probit employment policy_shock education age if men==1
margins, dydx(*) atmeans post
Model 2 Women: probit employment policy_shock education age if men==0
margins, dydx(*) atmeans post

[Clyde Schechter]

Don't do it that way. Do this instead:

probit employment i.men##(i.policy_shock ?.education c.age)
// IN ABOVE, REPLACE ?. BY i. IF EDUCATION IS DISCRETE, c. IF CONTINUOUS
margins men, dydx(*) atmeans
margins men, dydx(*) pwcompare

Note: I assume that policy_shock is a discrete variable and age continuous. If those assumptions are wrong, change i. and c. accordingly.

[Amruta Bagwe]

Thank you, Clyde. I used these codes, and it worked for the probit model. However, I have another model which is a bivariate probit that looks at school and work outcomes for children. Again I want to check the impact of the shock along with other controls. Will the codes for checking the difference in the marginal effects estimate be different? Because I tried the same codes with the bivariate but the model was not converging.

[Clyde Schechter]

The approach shown in #2 is applicable to any regression model. Convergence is a separate issue. It is possible that the interaction model is for some reason failing to converge when separate models for each sex would converge. It is also possible that one or both of the sex-specific models would fail to converge. Dealing with convergence problems is difficult and, unless they turn out to be due to bad data, usually require simplifying the model.

So examine the iteration log from your model. Usually with non-convergent models the log likelihood reaches a plateau and keeps repeating, or cycling around in a small range. Identify the iteration number at which this begins and re-run the model, adding the -iterated(N)- option, replacing N by a number just 1 or 2 less than where the log likelihood stalls. Stata will then re-run the calculation up to that point and then provide interim results. Those results are not valid and you should not treat them as if they were. But they may show clues to the cause of non-convergence. Look for coefficients or standard errors that are absurdly high, or standard errors that are absurdly close to zero. Any variables exhibit that kind of pathology are likely causes of non-convergence. If you find any such, the first thing I would do is check the data to make sure that the data are correct. If the data is OK, then the next thing I would consider is whether the scale of the variable is similar to the scale of other variables in the model. Models sometimes fail to converge because they contain variables on very different scales--rescaling is the solution to that problem.

If you don't find any pathological variables, then you can try running the analysis with the -difficult- option. If that doesn't work, try using a different maximization technique. (See -help maximize- for the range of possibilities here, and add the corresponding -technique()- option to your command.)

If none of that works, I think you will have to start re-building your model. This is a trial-and-error process where you start with a model containing only your most important predictor variable. If it runs, then you add your next most important one and try again. You keep going with this until you finally reach non-convergence. Then you know that the last variable you added is problematic and must be omitted. You can then try to add in more of the remaining variables, again, one at a time. You will end up with a simplified version of your model that is as large as possible but still converges. If that model makes substantive sense, you can go with it. If not, then I'm afraid you are stuck. Or at least, I don't know of anything else you can do--if somebody else does, it would be great if he or she jumps in.

[George Ford]

webuse school, clear

biprobit private vote logptax loginc years
lincom [private]logptax - [vote]logptax

[Clyde Schechter]

Either George Ford or I misunderstands the question in #3. My understanding is that O.P. wants to contrast the marginal effects (not a probit coefficient) of some independent variable in a -biprobit- model on both outcomes in two subsets of the data defined by a dichotomous variable. The code in #5 does something completely different: it contrasts the probit coefficient of an independent variable on the two different outcomes in biprobit.

[George Ford]

If its biprobit, then it seemed to me the test was going to be across models. Maybe not.

You can get the margins and then test across models too.

If it's just within a model or sample, then go with Clyde's recommendation.

biprobit requires slightly different coding however as you have two models and you need to pick which one you are using.