Interactions and margins discrepancies

[Davide Cartagini]

Hi everyone (and good holidays for those of you who will reads this soon after I post this),

I've got into a problem which I cannot understand where its origins lay on. Put it simply, I've run the same model with two different syntaxes in STATA. Here the two codes:


* Syntax A
quietly forvalues v=1/40 {
reg ser i.pser##i.cohort3 ib5.edulvl i.sex i.year [pweight = cs_weight] if sampleA==1 & country==`v', r
estimates store reg_results
margins cohort3, dydx(i.pser)
estimates store margins_results
outreg2 margins_results using DESOcohSL, excel dec(2) ci keep(i.pser#i.cohort3) ct(`v')
}

* Syntax B
bys cohort: reg ser i.pser ib5.edulvl i.sex i.year [pweight = cs_weight] if sampleA==1 & country==2, r
bys cohort: reg ser i.pser ib5.edulvl i.sex i.year [pweight = cs_weight] if sampleA==1 & country==3, r
bys cohort: reg ser i.pser ib5.edulvl i.sex i.year [pweight = cs_weight] if sampleA==1 & country==4, r
bys cohort: reg ser i.pser ib5.edulvl i.sex i.year [pweight = cs_weight] if sampleA==1 & country==5, r
etc...


ser & pser are dummies. Edulvl is categorical. All the other variables are controls. The variables are correctly specified (i.e. I've encoded what to encode and cleaned what to be cleaned).
I cannot understand why the two syntaxes leads to different estimates for i.pser#i.cohort3. The results from syntax A are analytically impossible (out of expected 0-1 range for linear probability models), while the results from syntax B are in line with previous works and literature. Trying to troubleshoot this, I've got to the conclusion that the problem (probably) lays in the margins command. I've also noticed that syntax A is way slower than B.
Again, syntax B fixed the problem and now I have the estimates I need, but I'm curious to know why I've got those estimates, does anyone have an idea?

Thanks to you all in advance and have nice holidays!

[Clyde Schechter]

Well, syntax A and syntax B are both legal, but they do two different things. So let's go over what each one does, and then you can decide which one is a correct answer to your research question. (You should make that decision based on understanding what each of them does, not on whether you like the results each one gives!)

Syntax B is actually simpler. Each of the individual commands shown restricts the sample to a single country and then calculates a separate regression of ser on i.pser, with covariates, for each cohort in that country. The coefficients of the i.pser variables are then the differences in the expected values of ser for the value of pser from the expected value of ser for the reference level of pser. Moreover, the effects of pser (and all the other variables) for each cohort are estimated using only the data for that cohort, because you used -by cohort-. Consequently, if the covariates' distributions differ among the cohorts, the resulting estimates of marginal effects are only partially adjusted for the covariate differences, that is, they are adjusted only for variation of the covariates within the single cohort. These marginal effects are, accordingly, known as conditional marginal effects.

Syntax A also loops over the 40 countries in a similar way. And for each single country it calculates a regression of ser on i.pser##cohort3, with the same covariates specified in syntax A. I will assume that cohort3 is the same variable as cohort in Syntax B and that you are just relying on Stata's automatic recognition of abbreviated variable names. (If it isn't, well, that is a source of difference, right there.) Now -margins- comes into play. The marginal effects it calculates are a different thing from what the coefficients derived from Syntax B are in a couple of ways. First, they are not relative to a base category of i.pser: you get a separate, independent, marginal effect for each level of pser. That alone, is a big difference. But there is an even bigger difference on top of that. -margins- uses the entire estimation sample including all of the cohorts, to estimate the marginal effects of pser in each cohort. What it does is, for each cohort, it temporarily replaces the value of the cohort variable in the entire estimation sample, with the value of the cohort currently of interest, but leaves the other variables as they were, and then calculates the marginal effect. So if the distributions of the covariates differ among the cohorts, -margins- provides a marginal effect estimate that is fully adjusted for those differences in the covariates across the entire estimation sample. So the marginal effects calculated here are known as unconditional marginal effects, though usually we just say marginal effects without further specification.

So you need to decide whether you want full or partial covariate adjustment for your marginal effect estimates and choose accordingly.

[Clyde Schechter]

Partial correction to #2. I misspoke in saying that the results from margins would not be relative to a base category. That is incorrect--I was confusing the present situation with the estimation of predictive margins. But it is not true for marginal effects of a discrete variable: those are always relative to a base category in -margins-.

The remainder of what I said, however, is correct. The key here is that a different estimation sample is used between unconditional and conditional marginal effects calculations.