Why do many economists put an asterisk next to *p<.10?

[paulvonhippel]

In most fields, an asterisk is reserved for *p<.05, but in economics it is common to put an asterisk next to *p<.10, which is the default in the SSC command outreg2.

How did economics come to have a different convention than other fields? I have heard that this decision was never really discussed but crept in to econometric software starting with TSP in the 1970s. But I've only heard that from one person, and I'm not sure if there are other origin stories.

I'm hoping to avoid the deeper question of whether p values and thresholds are a good idea. That has been exhaustively discussed elsewhere. I'm just trying to understand how the convention came to be different in economics.

[Nick Cox]

Do you want to avoid the question (whether deep or shallow) of whether stars are a good idea given that P-values are shown? Here's a vote of No.

[George Ford]

TSP got it from somewhere. As I understand it, Fisher is largely credited for the * thresholds, and said “if P is between 0.1 and 0.9 there is certainly no reason to suspect the hypothesis tested. If it’s below 0.02 it is strongly indicated that the hypothesis fails to account for the whole of the facts. We shall not often be astray if we draw a conventional line at 0.05.’’ So 5% became the standard, with 10% at the extreme.

There's a movement away from the *, but I'm not a fan of it. When perusing research, it's nice to have some sort of indicator regarding precision and the null, especially when the null of 0 is interesting. But, that could be merely the consequence of me being used to it.

That said, it appears from Statalist that professors aren't teaching students not to p-hack. Posts here are loaded with p-hacking. On those grounds, I see the point of getting away from it.

[John Mullahy]

Humorous title; serious paper.

https://www.aeaweb.org/articles?id=10.1257/app.20150044

[Joao Santos Silva]

I used TSP from 1985 to 2020 (and still miss it), but never seen any stars because by default they are not printed. A quick look at the help file (yes, I still have it installed!) tells us that

STAR1= upper bound on p-value for printing at least one star (*), when STARS option is on. The default is .05. There can be up to 5 pairs of (STAR1,STAR2) values, which can apply to different sets of diagnostics. This option only applies to the diagnostics listed for the REGOPT command.

STAR2= upper bound on p-value for printing two stars (**), when STARS option is on. The default is .01 . This option only applies to the diagnostics listed for the REGOPT command.

STARS/NOSTARS indicates whether stars should be printed indicating significance of diagnostics. STARS implies PVCALC, except for regression coefficients (@T).

So the default is one star for 5%, so I do not think TSP can be blamed for it.

[paulvonhippel]

Thanks for checking that! I don't think I've ever met a recent TSP user before.
I wonder where the *p<.10 convention came from, then, and why it's the default in many economics journals and outreg2.
It's not a great convention, since with multiple hypotheses being tested, using p<.10 without correction substantially increases the rate of false discoveries.
We make the argument starting on page 14 of this working paper. Below is a key figure. I still don't know where this convention came from and why it's different in economics than in other fields....

[Nick Cox]

I don't think it's been established that economics is different here. But it is hard to know better without some massive search exercise hunting through different literatures.

I think there is a massive cultural variation in modelling style. Most of my research focus has been in various Earth and environmental sciences where even though there is massive uncontrolled complexity and variability researchers still feel an influence from physics as the lead or exemplary science. What follows is overwhelming emphasis on simple models with few predictors. I wouldn't want to try to publish anything with more than about 3 predictors.

Conversely, what are much better represented in Stata circles are economic and social problems on the one hand and health or medical problems on the other. Here two stances are common: many candidates predictors plausibly have some effect, even if small, and (linked to that) a predictor is often better included in the model any way so that it has its chance, you can show that you have considered it, and you're not being arbitrary in excluding something. I am not even going to mention interactions, including quadratic effects.

So, what follows is ritual display of output for very often 30 or 100 predictors. So, the tension is then simple if tacit. Look, in principle, beloved readers, I am giving you all the information you need about my model fit. In practice, I know you are all busy people and some of this is just tedious or boring, so I am starring the bits of my model that appear particularly important.

[Nick Cox]

I see that massive was evidently my word of the day yesterday. Sorry to anyone who noticed undue repetition and flinched at a failure of style.

[Maarten Buis]

Nick Cox I don't think that that is a massive problem