Help with multilevel model to analyse change over time

[Elena Baro]

Dear Stata Users,

I am analysing the evolution of political attitudes (such as trust in politics, support for economic redistribution, satisfaction with the functioning of democracy, euroscepticism...) over time and how they relate to the preference for populist parties, using the European Social Survey data. The central question that I am trying to respond to is the following: do these attitudes have the same impact at different time points, or do they matter more (or less) in specific years (such as in times of crisis)?

I am analysing 9 countries, and I have 9 time points. The ESS data have modules, that are repeated, through stable design and mostly unchanged contents in every wave of the survey (even though with different respondents for each wave).

I am currently using a multilevel model to track the change over time and the different impact that the attitudes have in the analysed years. I have now landed on a model but I would like to get your feedback on whether it is correct or not. The dependent variable is binary, expressing the preference for populist parties (1) or mainstream parties (0). The time variable (essround) goes from 1 to 9. The independent variables are continuous variables which are the results of combined items after checking for Cronbach's alpha. I also have a series of control variables such as age, education level, occupation, religiosity, income, gender, left or right ideology.

melogit populistvote trustpolitics satisfaction socialint euroscept immigration_good ecored pdwrk income edulvla lrscale rlgdgr agea gender_male i.essround [pw= pspwght] || Country: essround, cov(unstr)

As the model is already pretty complex, I was thinking about running a series of models with only the key predictors (trustpolitics satisfaction socialint euroscept immigration_good ecored) and the interaction term with the time variable essround.

 melogit populistvote i.essround##c.trustpolitics  || Country: essround, cov(unstr)

From this, I was planning to calculate the marginal effects to see the changing effect that the specific attitude has at different time points in determining the preference for populist parties:

 margins, dydx(trustpolitics) over(essround) atmeans nose

margins, at(trustpolitics =(1/10)) over(essround) atmeans nose

Do these codes look correct for the purpose of the analysis?

Also, is the margin code suitable if I want to calculate the predicted probability to support populist parties per each predictor, at different time points?

Thank you,

Elena

[Clyde Schechter]

I have now landed on a model but I would like to get your feedback on whether it is correct or not.

Although it has become a cliche, it remains true that, as Box said, all models are wrong but some are useful. Whether this is a useful model is a substantive question, not a statistical one, and you need to run this by people in political science for their opinions.

One aspect of modeling that is statistical, at least in part, is whether the random effects model is appropriate here. Nine countries is a rather modest sample of country-space. I won't go so far as to say it is too small altogether, but my inclination would be to not use a multi-level analysis but instead do a conditional logistic regression (-xtlogit, fe-) with Country as the conditioned-on fixed-effect. Now this approach would preclude getting random-slopes for essround at the Country level--but see my remarks below about that questioning whether you really need to do that. If you do, then you must stick with -melogit-

Assuming for purposes of discussion that the underlying model is sound, here are my thoughts about the code you show.

The code for both -melogit- commnds makes a fundamental error by including i.essround in the bottom level of the model and then having a random effect on essround (without the i.). The problem is that without the i., essround is interpreted as a continuous linear time trend variable. Yet it has the same name as the discrete i.essround in the bottom level of the model. The -melogit- commands themselves will probably run without error messages but will be producing incorrect results. When you get to -margins-, Stata will protest that it cannot make sense of this and will terminate execution without results.

So you have to decide whether you want to have essround as a discrete variable or a linear time trend. Based on your description of your research question in terms like "do they matter more (or less) in specific years (such as in times of crisis) [emphasis added]" it sounds like a discrete variable is appropriate and a linear time trend is not. But the fact that your coding is ambiguous suggests that you may not have actually made a conscious decision about this. You should do so before running anything. Assuming you want a discrete representation of time, you have to use i.essround in both places. (If you settle on a linear time trend, then use essround with the c. prefix in both places. (Note: -mixed- accepts factor-variable notation in the random effects portions of models in versions 16 and 17. When -mixed- was first introduced, however, it did not. I don't remember when this became possible. Suffice it to say, I know it is allowed in version 16 and beyond, and do not remember further back than that. If you are using version 15 or earlier, you may have to resort to using the otherwise-obsolete -xi:- prefix to handle the indicators and interaction terms, and you will be unable to use -margins- thereafter.)

The next question is why you are including a random slope for essround (or i.essround) at the Country level. I'm not saying it's wrong, but it has nothing to do with anything you described about your project's goals. By including it there, you are asserting that the effect of that time variable on the probability of populist voting is differs across countries and calculating estimates of the country-specific effects. That may or may not be reasonable--again, that is something that a political scientist, not a statistician, should advise you on. I'm just pointing out that you never said that in words, so I'm wondering why you put it into code.

Turning now to the -margins- commands, you probably should not use the -over()- option. It is more likely that you want -margins Country, dydx(...- rather than -margins, over(Country) dydx(...-. These will produce different results, and in most circumstances it is the former, not the latter, that is intended. The difference is that -margins Country...- produces results that are adjusted for the confounding effects of the other model variables, whereas the results from -margins, over(Country)...- are not. Since you went to the trouble of putting those covariates into the model, it would be surprising to then discard those adjustments when calculating Country specific marginal effects. There are research questions where the unadjusted results are the correct answer, but they are unusual and nothing you have said about your research goals suggests that yours is one of them.