Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Post-hoc power analysis

    If I have 2 independent populations with means and standard deviations, how can i calculate the power of that test with a specific difference in mind that is not the observed difference?

  • #2
    For two independent samples, you may compute the power for a two-sample test of equality of means using sampsi (type help sampsi). But, as you may know, this is generally not a good idea. For one thing, many reviewers have learned (with considerable justification) that "post-hoc power analyses are wrong," which means that you are unlikely to be able to refer to the results of the calculation in a publication. Moreover, post-hoc power calculations make no sense from a frequentist viewpoint, assuming that you have performed a frequentist analysis of the data. Perhaps most importantly, such calculations are typically uninformative—they typically show that you have low power in cases where your results are not statistically significant, and that you have high power in cases where your results are statistically significant.

    As an alternative, it is generally better to use a confidence interval for the difference in means (e.g., as provided by ttest) to determine the range of possible differences with which the data are consistent.

    Comment


    • #3
      your topic title and your text appear somewhat inconsistent to me; if you are really interested in "post hoc power", then (1) I think it is a bad idea and (2) you might want to take a look at the "advice" section of the following page: http://homepage.stat.uiowa.edu/~rlen...~rlenth/Power/

      however, you seem to be inquiring about the power for some non-observed result based on means and standard deviations actually observed - did you do a pilot study and are now interested in the power for a difference other than the one seen in the pilot? (or, similarly, do you have data from some other study that you think might be useful in planning your own?); at any rate, you can use -power- (see -h power-) which is the name in the current version of Stata for what Phil pointed you to

      at any rate, for better advice you need to be clearer about what your situation is and what you want to do

      Comment


      • #4
        Originally posted by Rich Goldstein View Post
        at any rate, you can use -power- (see -h power-) which is the name in the current version of Stata for what Phil pointed you to
        Indeed—thanks, Rich, for the correction. I am of course familiar with the (relatively) new power command, but have not yet erased all of the muscle memory associated with the old sampsi.

        Comment

        Working...
        X