Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Unobservable Selection(Oster 2016) Using psacalc

    Dear All,

    I am recently learning Oster(2016)'s approach for robustness checks with respect to unobservable selection bias. Specifically, I am using psacalc for this practice. And I have two questions when implementing this:

    (1) Negative Delta
    When calculating the delta for beta=0, the command I use is:
    psacalc delta variable_of_interest

    and I got values of delta being negative (-0.3 and in some regressions -4). But as Oster(2016)'s major analysis is taking care of the situation where delta is assumed to be positive, so I'm confused about the output and how to interpret the negative delta case.

    (A similar post is: http://www.statalist.org/forums/foru...e-of-selection, and it is not answered yet. )


    (2) Use psacalc with DID setup
    Now I'm trying to use a DID specification with psacalc. My regression is the following:
    areg Y_ct Treatment_c*Post_t Post_t, absorb(ID_c)

    Here the treatment is at group level c and is continuous(like a dosage). Post_t is a dummy indicator indicating post-treatment period, and ID_c is group identifier. With the DID specification, I've controlled the group FE and post dummy, and the variable of interest now is an interaction term: Treatment*Post.

    So when using psacalc to calculate the adjusted beta of this interaction term, I guess we should use: "psacalc beta Treatment*Post, mcontrols(ID_c, Post Dummy)", instead of "psacalc beta Treatment*Post"?

    (Is there any examples, e.g. in papers, that have used this approach in DID or RD settings?)

    Many thanks!!!

    Lingwei



  • #2
    From section 3 of Oster's paper, you'll see that \[\delta \frac{\sigma_{1x}}{\sigma_1^2} = \frac{\sigma_2^x}{\sigma_2^2}]. A negative delta means that if the observables are positively correlated with the treatment, the unobservables have to be negatively correlated with the treatment to get the beta you specified.










    Jorge Eduardo Pérez Pérez
    www.jorgeperezperez.com

    Comment


    • #3
      OK, I see. I was confused with this because when calculating bounding statements about β in Section 3, there's an assumption that the proportional selection is positive.

      Thanks again for your help Jorge!!

      Comment


      • #4
        Hello,

        I hope it is okay that I reply to this thread even if my question is not directly related, but since I understand that Jorge Perez Perez has helped written the psacalc command I am hoping this is a good place to start.

        I am also running a Diff-in-Diff regressions with a triple interaction term, to see if the treatment effect differs by gender. My questions are therefore the following.

        1. When running the Oster test, should I include both the triple interaction term (Post*Treated*Female) as well as the 'normal' diff-in-diff term (Post*Treated) in the uncontrolled regression if I am interested in the stability of the triple interaction term?

        2. Is it possible to apply the Oster test to the stability of the SUM of the coefficients? Whilst I understand this is probably not possible with the psacalc command, I was nevertheless wondering whether I could calculate delta manually for the sum of the coefficients, or whether this is inappropriate.

        I hope you can help!

        Kind regards

        Comment


        • #5
          Jorge Eduardo Perez Perez or others: Just for completeness: How do you interpret a delta from the psacalc command that is positive but close to 0, e.g. delta = 0.04? Is it true to say that the unobserved variables play a less important role in bringing the estimated coefficient to zero? Thanks a lot!

          Comment


          • #6
            A value of delta close to zero basically means that the treatment variable is almost uncorrelated with the unobserved variables, and thus OLS should be (almost) unbiased. You might find a brief discussion of Oster's method in our forthcoming Stata Journal article interesting (particularly pages 31-33):
            https://www.kripfganz.de/stata/

            Comment


            • #7
              Thanks a lot for your reply Sebastian! But now I am a bit confused

              I´ve read the paper "The Slow Road from Serfdom: Labor Coercion and Long-Run Developments in the Former Russian Empire" (The Review of Economics and Statistics, https://direct.mit.edu/rest/article/...rcion-and-Long) and there the authors emphasize that their delta is larger than 1, i.e. "values exceeding 1 imply that selection on unobservables must be significantly stronger than selection on observables to explain away our result. Indeed, the δ values that we compute are consistently larger than 1. ... These findings imply that a bias of our estimates by unobservables is unlikely and suggest a causal interpretation of the effect of serfdom on contemporary development".

              So do I want my delta to be smaller than 1 or larger than 1 (so 0 > delta > 1 or delta > 1)? Or is it just that values of delta around e.g. 0.5 are considered "problematic" ?

              Regarding the original question, I get that cases where delta is below 0 just means that unobservables and observables would have to be negatively correlated. So a delta with -1 < delta < 0 should be similar in interpretation to a delta with 0 > delta > 1 I guess. Similarly, delta > 1 and delta < -1 should have similar interpretations.

              Thanks again!
              Last edited by Leon Schmidt; 01 May 2021, 11:14.

              Comment


              • #8
                My previous answer might have been misleading. Without seeing the actual Stata command you typed, I might have misinterpreted your question. I now assume you have typed psacalc delta ... without the beta() option (or equivalently with the beta(0) option). I further assume that the OLS beta coefficient is significantly different from zero. If that is the case, then an implied delta of 0.04 would be a huge problem because it implies that already a very small selection on unobservables has a substantial effect of the estimates. If delta instead would be large, then your results would be "robust" (in the sense that they remain significantly different from zero, although the magnitude might change nonnegligibly) to a substantial degree of selection on unobservables. In that regard, I tend to disagree with the conclusion in the paper cited by you that a bias of the estimates is unlikely. Technically, a bias already arises with a small degree of endogeneity. The question is, whether it is of a practically relevant magnitude. "Practically relevant" in the Oster approach seems to mean whether the effect remains different from zero. This appears to be a bit shortsighted, because changes in the effect size can be practically relevant even if the effect remains significantly different from zero. Side note: I personally also do not see why equal selection on observables and unobservables, i.e. delta=1, should be a relevant threshold.

                Interpreting the sign of delta is difficult. Local to zero, it reflects the sign of the correlation between the treatment variable and the unobservables. But as we show in our forthcoming Stata Journal article, delta as a function of this correlation has a discontinuity point. Also, the steepness of this function is different to the left and right of zero. Unless one is fixated on the threshold delta=1, I have a hard time interpreting a specific value of delta.
                https://www.kripfganz.de/stata/

                Comment


                • #9
                  Thank you very much Sebastian for this detailed explanation! I now understand the Oster bounds much better!

                  Comment


                  • #10
                    psacalc
                    Best regards.

                    Raymond Zhang
                    Stata 17.0,MP

                    Comment


                    • #11
                      Dear all,

                      I need your help!!

                      I am very confused about the interpretation of the psacalc command output.

                      if we follow the steps:

                      sysuse auto, clear
                      regress price foreign mpg weight headroom trunk
                      psacalc delta weight
                      we will have as result delta = 0.30310

                      . psacalc delta weight
                      Bound Estimate ----

                      delta 0.30310

                      Inputs from Regressions ----
                      Coeff. R-Squared

                      Uncontrolled 2.04406 0.290
                      Controlled 3.78137 0.526

                      Other Inputs ----

                      R_max 1.000
                      Beta 0.000000
                      Unr. Controls



                      So our Delta is less than 1...is that mean that the model is robust? if so why in other documents we talk about a Delta greater than 1 ???


                      Finally is Delta with psacalc command is it the same of lambda with rcr command??

                      Thank you very much for your help.

                      Comment


                      • #12
                        Krauth's lambda is closely related to Oster's delta, but they are not the same. Personally, I find both very difficult to interpret. Please see a discussion in the following paper (in particular starting page 31):
                        https://www.kripfganz.de/stata/

                        Comment


                        • #13
                          Thank you very much Sebastian for your help! It's very appreciated.

                          Comment


                          • #14
                            Hi everybody,

                            It seems that Anna Jongma question in #4 wasn't answered yet. Since I am also interested in it, I would like to push it again. Specifically, I would be interested in an answer to her first question (how to combine the Oster bounds analysis with interaction terms). I'll copy her question here:

                            "I am also running a Diff-in-Diff regressions with a triple interaction term, to see if the treatment effect differs by gender. My questions are therefore the following.

                            1. When running the Oster test, should I include both the triple interaction term (Post*Treated*Female) as well as the 'normal' diff-in-diff term (Post*Treated) in the uncontrolled regression if I am interested in the stability of the triple interaction term?"

                            Does anybody know the answer? Maybe Sebastian Kripfganz or Jorge Eduardo Perez Perez ?

                            Thanks a lot!

                            Comment


                            • #15
                              To add to my question from above on how to combine interaction terms with the Oster bounds: I would estimate the regression with the main effect and interaction term, then estimate the same regression plus the control variables, and then separately estimate the psacalc for the main and interacted coefficient. When, e.g., estimating it for the main coefficient, I would keep the interacted term in mcontrol(), and vice versa. Would that be correct? Also adding Arthur Morris here as he also answered questions regarding this package. Thanks again!

                              Comment

                              Working...
                              X