Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • How can we choose the appropriate significance level to interpret based on sample sizes?

    With the large sample size, we normally somehow interpret the coefficient significant at 10%, but in the small sample size, we normally ignore the 10% significant level. The reason is that with the large sample size, 10% significant level still makes a substantial contribution.

    I am wondering is there any reference or justification about the sample sizes and significance level being noticed? For example, under 1000 observations we just focus on p-value<0.05, or over 100,000 observations, we can focus on p-value <0.1 (just my example to clarify my question).

    Much appreciated.

  • #2
    Phuc:
    I do not recall any reading that proposes your first statement (but it may well be my fault).
    The issue is that, with large sample size, even a hiccup more or less can make a statistical significant difference. However, ths difference might not have any meaningful effect on the variable you're measuring.
    That's why p-value totemism is a matter of endless debate/criticism.
    Googling with the key words -ASA p-value- gives back some interesting entries.
    Kind regards,
    Carlo
    (Stata 19.0)

    Comment


    • #3
      Originally posted by Carlo Lazzaro View Post
      Phuc:
      I do not recall any reading that proposes your first statement (but it may well be my fault).
      The issue is that, with large sample size, even a hiccup more or less can make a statistical significant difference. However, ths difference might not have any meaningful effect on the variable you're measuring.
      That's why p-value totemism is a matter of endless debate/criticism.
      Googling with the key words -ASA p-value- gives back some interesting entries.
      Thanks Carlo, I did read it, a very interesting document

      Comment

      Working...
      X