Fixed or random effects for control variables in a multilevel model?

[jorge coelho]

I am trying to fit a multilevel model with one explanatory variable at level 1, 10 control variables at level 1, plus 5 variables at level 2. Should the control variables be modeled as fixed or as random effects?
Help will be highly appreciated!!

[Chris Boudreaux]

There are a few things to mention here. First, the terms fixed and random effects have different meanings in the literature on multi-level models than they do in other panel data settings. In a multi-level model, fixed and random effects typically refer to whether the parameters are considered fixed or random. In the frequentist statistic approach, parameters are fixed--there is a true value that can hopefully estimated. However, some parameters, like slopes and intercepts can be random. These are the random effects in the multi-level model. In my experience, either level 1 or level 2 variables can be treated as fixed or random. But, since you are asking about control variables, I would specify them as fixed. Consider something like below (I'm assuming your dependent variable is continuous):

mixed dv iv controls || id:

The above will estimate a mixed effects model (i.e., a random intercept model). If you want to model any variables as random, then you need to specify a random slopes model, which is the same as above except you add these variables after the colon:

mixed dv iv controls || id: control1 control2

[Chris Boudreaux]

I apologize. I should have clarified in the above. dv = dependent variable; iv = independent variable; controls = control variables.

[Jeff Wooldridge]

I have to admit to being baffled by the popularity of mixed effects models as implemented by the mixed command. All unobserved heterogeneity -- the so-called "random effects" -- are assumed independent of the covariates, whether the variables of interest or the controls. Therefore, regular OLS is consistent for the average partial effects. Mixed can only be justified on efficiency grounds, and then one should cluster the standard errors at the highest level in order to obtain valid standard errors.

It's much more convincing to put in dummies at the higher level and drop the controls at the higher level; this accounts for systematic differences in the level one covariates at the second level, and has a better chance of uncovering causality.

In this particular example, what are levels 1 and 2?

If you decide to go the mixed route, I see no reason to allow random coefficients on any of the control variables.

[jorge coelho]

Thank you for the precious remarks. Previous studies have adopted a mixed effects model, so this will be the choice. The DV is job satisfaction and the IV at level 1 is work life confict (several controls at level 1, including age, gender, work hours, time pressure, ...). The slope for work life conflict is moderated at level 2 by country culture (6 dimensions).

[jorge coelho]

There are a few things to mention here. First, the terms fixed and random effects have different meanings in the literature on multi-level models than they do in other panel data settings. In a multi-level model, fixed and random effects typically refer to whether the parameters are considered fixed or random. In the frequentist statistic approach, parameters are fixed--there is a true value that can hopefully estimated. However, some parameters, like slopes and intercepts can be random. These are the random effects in the multi-level model. In my experience, either level 1 or level 2 variables can be treated as fixed or random. But, since you are asking about control variables, I would specify them as fixed. Consider something like below (I'm assuming your dependent variable is continuous):

mixed dv iv controls || id:

The above will estimate a mixed effects model (i.e., a random intercept model). If you want to model any variables as random, then you need to specify a random slopes model, which is the same as above except you add these variables after the colon:

mixed dv iv controls || id: control1 control2

Would you have any reference for supporting controls as fixed effects??? Thanks!!!

[Jeff Wooldridge]

Jorge: How many countries do you have?

[jorge coelho]

Jorge: How many countries do you have?

About 23 countries ...

[Chris Boudreaux]

Would you have any reference for supporting controls as fixed effects??? Thanks!!!

I do not have any reference, but I agree 100% with Jeff--I see no reason to treat the controls as random coefficients given that you are only interested in them as control variables.

If I were you, I would form a hypothesis based on the random intercepts model and then explore whether you want your independent variable to have a random coefficient.

It sounds like you are in the psychology or management field, where these multi-level models are much more popular than in say economics so I understand the desire to use familiar methods, which you absolutely should do. However, keep in mind that this field is gaining appreciation for the often violated random effects assumption that Antonakis et al. (2019) has pointed out. Heeding this, you might consider Jeff's advice to estimate a fixed effects model (i.e., country-level dummies) as an additional alternative. This model is consistent even if the random effects assumption is violated.

Antonakis, J., Bastardoz, N., & Rönkkö, M. (2019). On ignoring the random effects assumption in multilevel models: Review, critique, and recommendations. Organizational Research Methods, 1094428119877457.

[jorge coelho]

I do not have any reference, but I agree 100% with Jeff--I see no reason to treat the controls as random coefficients given that you are only interested in them as control variables.

If I were you, I would form a hypothesis based on the random intercepts model and then explore whether you want your independent variable to have a random coefficient.

It sounds like you are in the psychology or management field, where these multi-level models are much more popular than in say economics so I understand the desire to use familiar methods, which you absolutely should do. However, keep in mind that this field is gaining appreciation for the often violated random effects assumption that Antonakis et al. (2019) has pointed out. Heeding this, you might consider Jeff's advice to estimate a fixed effects model (i.e., country-level dummies) as an additional alternative. This model is consistent even if the random effects assumption is violated.

Antonakis, J., Bastardoz, N., & Rönkkö, M. (2019). On ignoring the random effects assumption in multilevel models: Review, critique, and recommendations. Organizational Research Methods, 1094428119877457.

Thank you Chris for your insights!!! and for the reference!!