Dropping observations based on percentiles.

[Dillan Patel]

Hi,

I have a weekly wage variable (GRSSWK), and I want to drop the observations for the top 1% and bottom 1% of earners to remove any outliers. Does anyone know how to do this?

Best regards,
Dillan

[Clyde Schechter]

summ GRSSWK, detail
keep if inrange(GRSSWK, r(p1), r(p99))

That said, bear in mind that it is usually not a good idea to remove outliers. While you might want to examine outliers as possible data errors, and fix any errors found upon review, if the data are correct, it is usually problematic to just remove them for no reason other than that they are outliers.

[Dillan Patel]

Thank you for the response Clyde! For context, I am conducting a research paper to 'estimate the effects of trade unions on wages, and the corresponding impact on the low-paid'. I too initially planned to keep the outliers in my regression but then I had one individual with earnings of £120,000 per week and others with just £1 which seemed irrelevant. Do you still suggest I keep the outliers in my regression?

Best regards,
Dillan

[Maarten Buis]

You need to inspect the unusual observations and make a case by case decision on whether this is a data error or a genuine answer. If it is a genuine answer, then you keep it regardless of how unusual it is. If it is an error, then you fix it if possible. Only when that is impossible, do you consider dropping.

[Nick Cox]

Furthermore, this is one of many reasons why looking at log wages can make more sense than looking at wages. Still, those extremes do stretch credibility with different tales according to those tails.

[Dillan Patel]

Thank you all for the advice. I have decided to keep the outliers in and just make reference to them in my research.