[Clyde Schechter]

I wonder if I can say something like "In province D, a one unit increase in the score is associated with 91 percentage points higher chance of having mental health problems, whereas in province C, it is associated with only 34 percentage points higher chance."

Assuming these are outputs of -margins- with predictions of probability following a logistic (or probit) model, then yes you can say that. In the current version of Stata, the default prediction of -margins- after -logit-, -xtlogit, re-, or -melogit- is, in fact probability--so you would be OK. I believe the same was true in version 16, if memory serves. In some earlier versions, again if memory serves, the default prediction of -margins- after logistic models was xb, so you would have needed to specify -expression(invlogit(predict(xb)))- to get probabilities. If you look at -help whatever_the_regression_command_was postestimation- and then click on the -margins- link, you will be able to see what the default prediction is in your version.

[Meng Yu]

The Stata I am using is version 16. In the description of margins for xtlogit postestimation, it reads, “margins estimates margins of response for linear predictions and probabilities.” I believe this means the results I get are percentage points.

[Clyde Schechter]

If you are talking about -xtlogit, re-, the default is, in fact, probability and you are OK.

If you ran -xtlogit, fe-, you have a serious problem. -margins- gives you "pu0" as the default prediction. But this is seriously problematic. You should not use -margins- for anything other than -xb- after a fixed-effects logistic regression, and in that context, there is no valid way to convert from the -xb- metric to probability. See Joao Santos Silva's remarks at #6 at https://www.statalist.org/forums/for...on-and-margins'.

[Meng Yu]

I am using xtlogit random effects model. According to the help file, under re, statistic, “pr: marginal probability of a positive outcome; the default.” I am using margins, dydx command.

For another study I conducted, I used either version 15 or 15.1. I wonder if the results were also probabilities for the same regression and command. I don’t have access to that version anymore. Thank you.

[Clyde Schechter]

I don't have version 15 on my computer any more either.

But, I tried running an example using -xtlogit, re- followed by -margins, dydx()- under version control to version 15 and it produces the same results as without version control in version 17. Moreover, when I did it again with version control version 13, it produces different results. Now, version control is not a time machine, but it usually does correctly reflect how the same syntax was differently interpreted in earlier versions. So it is highly probable that in version 15, probability was also the default output of -margins- after -xtlogit, re-, whereas in version 13 it was not.

[Meng Yu]

Thank you very much, I feel much released now. Thanks.

[Meng Yu]

I have another question on how to choose reference group for variables. I was told by different people to choose the largest group or to make sure the group you are interested in not as the reference group so that you can interpret their results. I changed the reference group for two covariates yesterday in my xtlogit regression and that made the result of the predictor variable not statistically significant anymore. I suspect if it is because the reference group of one covariate is now very small and this covariate is related to the predictor. When the reference group was a category, its coefficient was not statistically significant.

[Clyde Schechter]

Remember that the coefficient of a level indicator for a categorical variable represents the difference between that level and the reference group. So if you change the reference group, the coefficients now have a different meaning. Suppose there are three levels, and suppose that group 2 is very different from group 1, but group 3 is almost the same as group 2. If group 1 is the reference group, then groups 2 and 3 will have large coefficients. But if you now make group 3 the reference group, group 2's coefficient will be close to zero and group 1 will have a large coefficient.

If you look at predicted margins instead of coefficients, you will find that they do not change regardless of which group is the reference. Similarly any linear combination of the form x.group - y.group will come out the same regardless of what the reference group is.

Putting it briefly, the coefficient that changed it's significance, has also changed its meaning. So no surprise and no worry. If you go back and calculate the difference between that coefficient and what is now the coefficient of the original reference group, you will see that that is unchanged from the original coefficient.

The choice of reference group is completely arbitrary, and is a matter of convenience. In general, we choose a reference group to be the one we are most interested in contrasting with so that the coefficients give those contrasts directly without further calculation. (For example, if there are three treatment conditions plus a no-treatment condition, we would ordinarily use no-treatment as the reference group.)

[Meng Yu]

I wonder if this is one reason why you have abandoned the concept of statistical significance.

I knew changing reference group of a variable would not change the relationship between the different categories. I wanted to know why changing the reference group of a covariate would change the statistical significance of another variable in the model, in this case it was a continuous variable.

[Clyde Schechter]

That should not happen, unless the model also contains an interaction between that continuous variable and the categorical variable whose reference level changed. In that case, the continuous variable has also changed its meaning. In a categorical#continuous interaction, the coefficient of the continuous variable represents the marginal effect of the continuous variable (on xb) when the categorical variable equals its base level. So if you change the base level of the category, the continuous variable's coefficient now represents something completely different.

If in your example, there is no interaction term involving the continuous and categorical variables in question, then something very peculiar has gone on--that shouldn't happen, and you should post both the commands and output for both analyses.

[Meng Yu]

I also found that was very odd. Will have to wait until tomorrow to have access to the data and software.

But, I tried running an example using -xtlogit, re- followed by -margins, dydx()- under version control to version 15 and it produces the same results as without version control in version 17.

I wonder if you could try version control to version 14.1 for me please. I just found out the version I used was 14.1. Or if you could tell me the command for me to try it myself. Thank you.

[Clyde Schechter]

Run your -xtlogit-. Then run your -margins, dydx()- command the way you have been doing it. Then run it with -version 14.1:- prefixed in front of it. If you get the same result, then you know that version 14.1 is doing it the same way as your current version. If you get a different result, it means that version 14.1 did something different (most likely, predicting -xb- instead of probability).

[Meng Yu]

Thank you very much. For a reason out of my control, I probably won't have access to the data and software until Tuesday. I wonder if you could try version control for 14.1 for me if it is not too much inconvenience. I just want to put my mind at ease to some degree. Thank you.

[Clyde Schechter]

Done. version 14.1 gives the same results as version 17, predicting pr.

[Meng Yu]

Perfect! Thank you very much! Now I only need to worry about why changing reference group of a covariate changes the result of the predictor variable.