Problem with the option "osample" from the command "teffects nnmatch"

[Juan Hernandez]

Dear Statalist,

My team and I are running a propensity score matching analysis on agricultural data in order to assess the impact of credit in several farmer outcomes. For this, we are using the psmatch2 command, but I was frustrated as we never got to completely and globally balance the treatment and control groups on the observable covariates considered. So we decided to switch to the command teffects nnmatch, that allows to perform the matching exactly on the observable covariates, which is useful for categorical variables for example.

Anyways, we are now getting an error after running the command, which is similar to the one of this thread:http://www.statalist.org/forums/foru...match-question

We first run a line of code that looks like the following:

teffects nnmatch (dep_variable x_covariate) (treatment) which gives the following error message: no exact matches for observation 23472; use option osample() to identify all observations with deficient matches
We then include the option osample: teffects nnmatch (dep_variable x_covariate) (treatment) , osample(newvar) After Stata identifies the observations for which an exact match cant be found (they have a 1 in newvar). We do the following:
teffects nnmatch (dep_variable x_covariate) (treatment) if newvar == 0 Unfortunately, we end up with an error message similar to when we first ran the command, that is, the Stata claims it didnt find exact matches for a certain observation, different than the one at the beginning. Each time I run it, it takes about 2 hours to finish, so I'm not sure how to deal with this issue without having to run several times the command with the option osample.
Does anybody know a solution for this problem?


Best regards,


Juan

[Juan Hernandez]

nobody??

[Jonathan Seiden]

I had a similar question to Juan's. Bumping this hoping it gets answered.

My situation is similar, where I have code as follows:

    
teffects psmatch (outcomevar) (treat $variables), caliper(0.05) osample(outofrange) vce(robust)

Which returns the message:

1 observation has no propensity-score matches within caliper 0.05;
it is identified in the osample() variable

So, like Juan, I wish to drop or exclude this obs from my psmatch, re-running as such:

drop if outofrange == 1    
teffects psmatch (outcomevar) (treat $variables), caliper(0.05) osample(outofrange2) vce(robust)

But I again get a similar message:

2 observations have fewer than 2 propensity-score matches within caliper 0.05; they are
identified in the osample() variable

I can repeat this procedure and drop the variables a few more times, but I don't understand why this should be happening and ideally I would like to automate this iterative process. I could certainly build a for loop, but it feels like I might be missing something important.

Thanks for the help!

[Juan Hernandez]

Hi Jonathan,

I had forgotten about this. If I remember correctly, maybe a solution can be to set a seed before running the teffects command, that way I think you can make stata drop the same observations always and avoid that issue.

Best,


Juan

[Jonathan Seiden]

Hi Juan, sorry for taking a year to respond. I had meant to update this a while back. I think I understand what the problem was. The model refits the propensity score each time you run it based on the multivariate logit. By dropping observations, the PS fit changes slightly and might find additional observations that are out of range.

The solution I found was to first fit the logit model for the propensity score as a variable, predict the propensity score, and then run the teffects command on just that propensity score.

I think the below should prevent running into the issues described above:

 logit treat $variables  
 predict ps  
 teffects psmatch (outcomevar) (ps), caliper(0.05) osample(outofrange) vce(robust)  
 drop if outofrange == 1    
 teffects psmatch (outcomevar) (treat ps), caliper(0.05) vce(robust)