Log restricted-likelihood in the multilevel model

[Jessica Rayse]

How to interpret the Log restricted-likelihood in the multilevel model? I am running multilevel models in which the Log restricted-likelihood is positive and increasing. Shouldn't the Log restricted-likelihood be negative and decreasing as the model improves in the step up strategy? Wouldn't the closer to zero the better?
The image below is the output of the unconditional model (without the insertion of explanatory variables). The model with random intercepts and the model with random slopes resulted in Log restricted-likelihood even higher than these.

[Clyde Schechter]

The belief that the log likelihood (restricted or not) should be negative is mistaken. These likelihoods are based on an underlying model that is based on probability density functions. Probability density functions are not probabilities, and can take on values > 1. For example, the probability density function of a normal distribution with standard deviation 0.1 peaks at about 4. Notice that the variances of your random effects are fairly small numbers: that implies that the corresponding density functions will be large numbers. Those get multiplied together to produce the (restricted) likelihood function.

[Jessica Rayse]

The belief that the log likelihood (restricted or not) should be negative is mistaken. These likelihoods are based on an underlying model that is based on probability density functions. Probability density functions are not probabilities, and can take on values > 1. For example, the probability density function of a normal distribution with standard deviation 0.1 peaks at about 4. Notice that the variances of your random effects are fairly small numbers: that implies that the corresponding density functions will be large numbers. Those get multiplied together to produce the (restricted) likelihood function.

Thank you very much for your explanation. But if possible, I would like some further help:

Note that in the model below, var1 is not statistically significant in the fixed components, but it is in the random component.

On the other hand, var2 is statistically significant in the fixed components, but it is in the random component only at 10%. In your opinion, how should I proceed with the strategic step-up since there is no stepwise procedure for multilevel models? Should I keep these variables?

[Clyde Schechter]

Full disclosure: I do not believe in building models by including/excluding variables based on statistical significance criteria. And I think nearly everybody who has given the matter much thought recognizes that stepwise regression is a bogus procedure. Take a look at https://www.stata.com/support/faqs/s...sion-problems/ regarding the latter.

Now, notice that in the random effects part of the model, no z or t-statistics or p-values are provided. Clearly they could have been provided since the estimate and standard error are both provided. The reason for that is that these variance component estimates do not follow normal distributions. And a null hypothesis test would be strange at best. First of all, as variances they cannot possibly be negative. So zero, which is a possible, but extremely unlikely value, is an edge case. The usual statistical tests simply would not work. No matter how you slice it, you cannot do a hypothesis test against a null value of zero in the way you have attempted. The appropriate way to do a statistical test for whether to retain or remove the random slope is to do a likelihood ratio test comparing the models with and without the random slope. (When you do that, the -lrtest- command will also report that the test assumes the null hypothesis is not on the boundary of the parameter space. But a random slope variance of 0 is the boundary of the parameter space. Consequently the hypothesis test is conservative: it will not falsely reject, but it may falsely fail to reject the null hypothesis.)

So, if you are going to use hypothesis testing to decide which variables to include in the model, use the correct test. Then, apply your decisions separately to the fixed effect and the random slope. It is fine to retain the fixed effect and remove the corresponding random slope, if that's what your decision rule tells you. It is also OK to remove the fixed effect and retain the random slope. That may seem wrong since we are usually taught to always include the fixed effect corresponding to any random slope. But the reason we are taught that is that by omitting the fixed effect, we are constraining the model to having that fixed effect = 0. But if your decision rule tells you to act as if the fixed effect is zero, then removing it from the model is just one way of doing that.

As I said at the top, I wouldn't use statistical tests for this purpose anyway. But whatever kind of decision rule you use for variable inclusion, just apply it separately to the fixed effect and the random slope parts of the model. And if you are using statistical significance tests for this purpose, use the right one for the random slope: the likelihood ratio test.

[Jessica Rayse]

I really appreciate your contribution. Thank you very much.