Should I Use Weights?

[Shisho Jakas]

Dear Statalist,

I have this design for my empirical work, and there is one thing I got confused about.

I have individual-level panel (household survey) data and convert it into community-level panel data to deal with some issues by averaging out the outcome of individuals.

However, each observation (community) has a different number of individuals combined (numppl) and it ranges around from 50 to 150.

To deal with this difference in the number, I use the weighting in Stata by adding [aweight=numppl].

However, OLS estimates gave lower power and magnitude compared to the one without weighting.

For the type of data and design for my research, I am not sure whether weighting is necessary or not.

I read some articles and papers about weighting, but it was not so clear for me to reflect it to my research.

Or should I as a control variable instead for village level analysis?

I would appreciate it if you could give me some advice on that. Thank you in advance.

[George Ford]

Weighting is confusing in econometrics. Generally, I think, it is discouraged. But it is done a lot.

You'd think of the difference as the coefficients measuring effects of communities when unweighted. When weighted, it is more like a person result.

What's happening, I suspect, is that there's not much going on with the larger communities, which is why the results get weaker.

When you collapsed the data, did you use survey weights?

[Clyde Schechter]

The idea behind aweights is that you weight each observation in inverse proportion to its error variance. When a value is created by averaging N observations of the same case, the error variance is 1/N times the error variance of a value based on a single observation of that unit. That's why the number of observations used to create the average serves as an aweight.

There is a theorem that says that if you have a series of estimates of some parameter, x1, x2, x3, ..., xn, each with different error variance, then the "best" (in some sense I no longer remember, under conditions I no longer remember, but probably mild ones) estimate of the parameter based on those estimates is the weighted average of the x's, weighted in inverse proportion to their error variance.

I agree with George Ford that the explanation for the difference in results is that most of the effect you observed in the unweighted analysis is coming from the smaller communities, with less in the larger communities.

[John Mullahy]

This paper by Jeff Wooldridge and co-authors is my go-to paper on the topic. (It also boasts possibly the best title ever to appear in the economics literature.) https://jhr.uwpress.org/content/50/2/301

[David Radwin]

Also see

• Bollen, K. A., Biemer, P. P., Karr, A. F., Tueller, S., & Berzofsky, M. E. (2016). Are survey weights needed? A review of diagnostic tests in regression analysis. Annual Review of Statistics and Its Application, 3, 375-392. https://www.annualreviews.org/doi/fu...-011516-012958
• West, B. T., Sakshaug, J. W., & Aurelien, G. A. S. (2016). How big of a problem is analytic error in secondary analyses of survey data?. PloS one, 11(6), e0158120. https://journals.plos.org/plosone/ar...l.pone.0158120
• Winship, C., & Radbill, L. (1994). Sampling weights and regression analysis. Sociological Methods & Research, 23(2), 230-257. https://scholar.harvard.edu/files/cw...ng_weights.pdf