Testing the economic significance of a variables based on -xtreg, fe-

[Claire Thompson]

I'm trying to find out if certain independent variables have a economically significant effect on my dependent variable. After my regression with -xtreg, fe- I used

mfx, varlist(age)

default predict() is unsuitable for marginal-effect calculation
r(119);

How should I calculate the economic effect according to you?

univar age

gives the standard deviation of age. To calculate the economic effect I believe that I must use the marginal effects and the standard deviation.

I also tried to find the solution but I didn't find it.

[Maarten Buis]

xtreg is a linear model, so the raw coefficients already contain all the information you want.

[Claire Thompson]

Maarten, could you please explain/elaborate that? I can only see if the independent variables are statistically significant or not based on the t-values and p-values of the independent variables

[daniel klein]

You are asking for marginal effects. In a linear model, the estimated coefficients are equal to the marginal effects, so there is no need to compute them.

Best
Daniel

[Claire Thompson]

Thank you Daniel.

My last question is what is the general method to test the economic significance?
I remember that I need to use the marginal effects (coefficients in the case of -xtreg, fe-) and the standard deviation but I don't know anymore if I need to multiply or divide them and if the outcome of that computation is simply a matter of subjective reasoning or objective reasoning to state that a independent variable has a economically significant effect or not.

A lot can be found about statistical significance, but not about economically significance

[daniel klein]

A lot can be found about statistical significance, but not about economically significance

That has probably to do with the fact, that there is no agreement on what exactly is "economic" significance.

A lot of people prefer some effect size measure, others (probably including yourself) like to report standardized coefficients as a measure of effect size. In my view, you are best of just interpreting the raw regression coefficients as these are (hopefully good) estimates of the causal effects of x on y, or at least they represent the relationship between the predictors and your response. Standardized coefficients in turn, are a mixture of the underlying relationship and the (observed) variance of the respective predictor (for a more detailed and entertaining discussion see King 1986, p.669ff.).

Best
Daniel


King, Gary (1986). How not to lie with statistics: Avoiding common mistakes in quantitative political science. American Journal of Political Science, 30(3): 666-687.

[Claire Thompson]

Thank you for the clarification Daniel.

[Maarten Buis]

Economic significance cannot be formally tested; it just means that the size of the effect is meaningfully large. So this is necessarily a subjective judgement call. The important thing is that you can interpret the size of the coefficients, i.e. that both the dependent and independent variable have a meaningful scale. After that you just stare at the results and think.

[Nick Cox]

In my reading the point about "economic" (or physical, biological, Star Trek, whatever) significance is that it is whatever you care about beyond the statistics of a problem.

Would you notice or care about an effect in practice?

Suppose I find that a difference of altitudes of 1 m is significant at conventional levels.

Do I care?

If it's sea levels, I really do.

If it's mountain top altitudes, I really don't.

The distinction lies in scientific judgment drawing upon scientific arguments and experience. Of course, if you don't work on altitudes, that might be hard, although I chose an example that should be clear beyond my field.

[Christian Agethen]

I assume you have more than one (economic) explanatory variable, keep in mind that a t-test is only suiteable for the individual significance of one variable. It is common in economics, that you want to test a subset of explanatory variables which then requires an F-test.

As Maarten and Daniel pointed out, "economic" significance might be a misleading term. Generally, you're interested in the significance itself (no need to be "economic") of an explanatory variable/parameter. Economists however tend to view the influences of an economic model as "causal", which is a bit more advanced and needs your model to fulfill additional requirements.