caliper size

[Michael Andersonnnn]

Please forgive me if I am asking a basic question.
I am experimenting with different approaches to PSM. Currently using nearest neighbour with and without replacement, and specifying my caliper size as 0.001. As follows:


psmatch2 ind alter sex i.volq i.charlindex i.imddecile , outcome(readmission) neighbor (1) caliper (0.001) odds logit ate common


psmatch2 ind alter sex i.volq i.charlindex i.imddecile , outcome(readmission) neighbor (1) caliper (0.001) odds logit ate noreplacement common


I am using a caliper size that is small as I have a very large amount of controls (300,000) for my treatment (120,000), therefore only a handful (few hundred) are off common support

My question is around what caliper size really means when using psmatch2. Is this % difference to propensity score? I see some literature around recommended caliper sizes of 0.1 or 0.2SD, but I am really not sure how to specify this
I have been through the psmatch2 help file and haven't found a clear answer to this question, any help would be much appreciated?

[Clyde Schechter]

I don't think there are any useful formulas or rules of thumb for selecting a caliper size. The purpose of matching is to reduce the bias in observational studies from confounding variables. In a world where unlimited amounts of data is accessible, you would not do caliper matching at all: you would always do exact matching and that would completely eliminate the confounding effects of the matched variables.

In the real world, if we insist on exact matching we end up with most observations failing to find a match. An analysis relying on only the matchable cases would likely be biased due to this very selection effect! So we would have defeated the purpose of matching. So, instead of requiring exact matching, we relax the conditions to allow approximate matching. If we set a very lenient limit, we will have fewer unmatchable cases, but it also means that we are not fully eliminating the confounding effects of the match variables, only partially so. So this is a trade-off.

In general, you want to keep the caliper small enough so that there isn't too much difference in the expected value of the outcome variable conditional on two observations being matchable within that caliper. So if the matching variable is strongly associated with the outcome, you need a very narrow caliper. If the matching variable is only weakly associated with the outcome, a wider caliper will still produce a reasonable approximation to exact matching. On the other hand, you also want the caliper to be wide enough that you aren't left with too many unmatched observations. I don't know of any algorithms that can identify acceptable calipers for a given problem. It is more often done by an intuition (perhaps informed by prior experience with these variables, or by exploratory analyses of them) about how large a difference between the matching variables you can get away with before the outcome variable differs by an unacceptable amount. That might give you a starting point. From there, trial and error until an acceptable compromise is found. I should add that, on occasion, there will be no acceptable compromise and matching simply won't be usable.

[David Radwin]

I don't know if it's useful or not, but one rule of thumb is to set the caliper to equal one-quarter of the standard deviation of the propensity score.

Stuart, E.A., and Rubin, D.B. (2008). Best Practices in Quasi-Experimental Designs: Matching Methods for Causal Inference. In J.W. Osborne (Ed.), Best Practices in Quantitative Methods (pp. 155–176). Thousand Oaks, CA: Sage.

[David Radwin]

So you could do something like this:

quietly logit outcome covars // estimate logit coefficients

quietly predict double pscore if e(sample) // generate propensity scores

quietly summarize pscore // calculate the standard deviation

local caliper = `r(sd)'/4 // calculate the caliper and save to local macro