[Carlo Lazzaro]

Roy:
I would:

testparm i(2008/2013).fyear

to investigate if they deserve to be kept in the model.

I would also say that you R-sq set is not that sky-rocketing: perhaps, a more parsimonious model can be considered.

[Martin Paul]

Dear statalist

I am facing a similar issue with my correlated random effect model.
I have three year dummies and 3regional dummies.
I understand STATA always drops one dummy to be used as comparison.
However, for my case, I loose 2 of my time and regional dummies.

What could be happening?
Here is my code:

 probit pdtndecision head_age  head_gen  totalinp dstfrcop head_edu wlkdsmnm chkpexpr totallab lrainfall cultarea lPrice_improved TLU  i.year i.district wlkdsmnmbar lrainfallbar cultareabar lPrice_improvedbar totallabbar TLUbar, vce(cluster qnno)

[Carlo Lazzaro]

Martin:
I would have started a new thread.
That said, did Stata report the reason for the omission (say, collinearity)?

[Martin Paul]

Yes Carlo
it said collinearity

[Carlo Lazzaro]

Martin:
omission due to collinearity is simply a matter of fact.
You can investigate why it occurred via -estat vce, corr- and then:
- let things as they are;
or
- consider a different specification of your regression model (consisten with the literature in your research field, i would add).

[Suborna Barua]

Dear Respected Members, I need your kind suggestion on my similar problem.

Hi, I am facing a similar problem with data from 1962 to 2014, largely unbalanced. I am estimating for eight regional panels and one aggregate panel with time dummies (using i.year). I am running PCSE estimation and it cannot estimate when I estimate casewise because of largely unbalanced data. It says:

"no time periods are common to all panels, cannot estimate disturbance
covariance matrix using casewise inclusion"


When I apply pairwise, it estimates but does not produce the coefficients and I get only dots. It says the following:

Warning: variance matrix is nonsymmetric or highly singular

However, when I mannually apply the time dummies one by one and remove some of the years from the time dummies, it generates the results. For example, if I use dummies manually as below, it estimates but I have to remove dummies for some years. For example, in the below estimation from year 1 to year 19 are excluded. If I add one more year, say year 19, the PCSE is giving me result as the above image, i.e., no coefficients, only dots. In addition, this number of dummies accepted also varies from panel to panel.

In this situation, I need your kind help and suggestion on these as I am seriously struggling:

1) Is it a problem If I report these results of different panels using different number of year dummies, using best possible set of dummies? And I explain in my paper that some year dummies are excluded because of unavailability of adequate data for those years and PCSE cannot calculate the variance matrix?

2) Is it a big problem if the number of dummies is less than the number of years in the panel regression (N)?

3) OR, any suggestion to fix it please?

[Mohamed Elsayed]

Dear Roy Steinvoort

Would you please indicate how did you fix your data so as to solve the problem of omitted year dummies? Or at least tell where was the problem in the data which led to the problem?

[Mohamed Elsayed]

Hi Clyde Schechter

I am conducting a data analysis. I have a panel with individual firms with firm-specific and 2 macroeconomic variables. I would like to run -logit- and robust standard errors adjusted for firm clustering and including years fixed effects (unbalanced panel).

My question is: when I include year-dummies will the effect of the "overall" macroeconomic variable absorbed by the dummies? in other words, will including year-dummies lead to multicollinearity between the year-dummies and the "macroeconomic" variables?

I am wondering because Stata omits 2 years dummies because of collinearity and the 2 macroeconomic variables are insignificant (they were significant before adding the year-dummies). When I rerun the model excluding the 2 macroeconomic variables, Stata doesn't omit any year-dummies. Remarkably, in both cases, the inferences are consistent.

Also, in this case:
1- Should I exclude the macroeconomic variables when including year-dummies? Or there will be no problem as macroeconomic variables become insignificant after this inclusion year-dummies?
2- Is there any way to overcome the omission of 2 year-dummies because of collinearity while running the model with both the 2 macroeconomic variables and year-dummies and the "macroeconomic" variables?
3- Did I fail to consider anything important here?

Thanks.

[Clyde Schechter]

Assuming that the macroeconomic variables take on the same values for all firms in any given time period (I take it that's what you mean by "macroeconomic"), then, yes, they will be colinear with the year indicators and something has to disappear, either the macroecnomic variables must be omitted, or two of the year indicators must be omitted.

From the perspective of inference about other variables in the analysis, and model outcome predictions, it makes no difference at all which variables you choose (or allow) to be omitted. The results will be the same, because the information conveyed by the two macroecnomic variables is already included in the information from the time indicators and vice versa. When you put them together, no new information is added to the analysis, and when you omit two of them, you lose no information, so everything else comes out the same.

Is there any way to overcome the omission of 2 year-dummies because of collinearity while running the model with both the 2 macroeconomic variables and year-dummies and the "macroeconomic" variables?

No. Because the macroeconomic indicators are 100% predictable from the time indicators (that's what colinearity means, after all) and vice versa, any model containing them all is unidentifiable. Therefore parameter estimation can only be accomplished after some constraints are imposed that identify the model. The conventional ways of constraining the model to identify it are to constrain some (as many as are needed) of the coefficients to be zero, which is to say, omitting those variables. You can contrive other constraints: you could stipulate for example that two different weighted sums of the coefficients have to be zero--then you would get estimates in which every variable would show up, but the estimates are subject to these two arbitrary constraints. Unless those constraints reflect some kind of "law of nature" or have a theoretical justification, the results are just arbitrary numbers. In fact, algebraically you can show that you can get the results on any two of the coefficients to come out to be any pre-specified values whatsoever, and the coefficients of all the other variables will automatically "adjust" accordingly. But clearly that is just number play, and not science. And, most important, none of these machinations affect any of the other estimates, nor the model outcome predictions. You can't get around linear algebra.

Should I exclude the macroeconomic variables when including year-dummies? Or there will be no problem as macroeconomic variables become insignificant after this inclusion year-dummies?

As already noted in the preceding paragraphs, it doesn't matter which of these things you omit. Everything else comes out the same. The reason I'm responding to this separately is because of the second phrasing of the question. The notion that variables can or should be omitted from a model because their coefficients turn out to be statistically insignificant is a wide spread but highly pernicious, even malignant, fallacy. You should make strenuous efforts to purge it from your brain. It is fine to leave out these macroeconomic variables; it is equally fine to leave out two of the time indicators. But it has nothing to do with what's statistically significant and what isn't. It's all because of the colinearity involving them.

Did I fail to consider anything important here?

That's a very difficult question. First, there may be all sorts of content-based issues that you have not considered, and I would have no way of knowing about them. Even purely within the realm of statistics, it's a very large field and I certainly don't have a command of all of it. I doubt anybody really does. More concretely, within any discipline there are statistical issues that are traditionally accorded great importance, issues that if not considered leave the analysis considered inadequate. Yet in other disciplines, that same issue might seldom if ever be considered. A good example is the use of the Hausman or similar test to choose between fixed-effects and random-effects regressions. From following Statalist I know this is considered an important and very standard practice in finance and econometrics. In my field, epidemiology, I can't remember ever seeing a Hausman test used--the choice of fixed or random effects modeling is done relying on content-based considerations. And whereas in finance and econometrics there is a strong preference for fixed-effects analyses unless there are explicit good reasons to use random effects, in epidemiology it goes the other way around. There may in fact be good reasons, related to the types of phenomena that get analyzed, why the standard practices and presumptions of the disciplines work so differently. My point, in any case, is that you really need to pose this question to somebody in your own field to get an adequate answer.

[Mohamed Elsayed]

Hi Clyde Schechter

Many thanks for your valuable explanations - this is really appreciated. Lastly, (I know this might be a trivial question) is there a problem of getting insignificant intercept? or Does getting insignificant intercept in the model indicate anything should be taken into account (everything else seems OK)? I read many in this and there is a consensus that there is no problem and it doesn't mean anything to care about. Nevertheless, I appreciate your opinion about that.

[Clyde Schechter]

I agree with the consensus. In nearly all situations, the constant term of a regression is of little or no interest. Moreover, in a fixed-effects regression, it isn't even meaningful. There it is an artifact of how you specify the fixed-effects themselves: change the base category and the constant term will change.

[Mohamed Elsayed]

Thank you so much, Clyde Schechter