Should I apply weights when I use the -tabulate- command? OECD PISA analysis

[Anders Astrup]

As is says in the title, I am unsure whether or not the analytic weights should be used when producing tables like the ones in the attachments below.

I am going to use tables like the ones below, to show how many girls and boys are in the different catogories. All of my analysis are for one country only, I do not include or compare observations from different countries.

Also I am doing quantile regression models, and for these I am sure I need to use the analytic weight "w_fstuwt": Final Student Weight.

In OECDs manual for secondary analysis, it seems to be advised to apply weights in all analysis.

Should I just go ahead and apply the weights to my tabulations as well?


Without weights:

With weights

Kind regards
Anders

[Clyde Schechter]

It depends on what the tabulations are for.

If your tables are intended to be a description of the sample data set, then no weighting is needed (or even appropriate). But if your tables are intended to be estimates of the distributions of the variables in the population your data are intended to generalize to, then you must apply weights.

[wbuchanan]

I would also say you may want to consider survey setting the data. The weights are likely a combination of weights to adjust for non-response bias and sampling probabilities. Analytics weights may not be appropriate. There should also be information on which method should be used to estimate the standard errors.

[Anders Astrup]

It depends on what the tabulations are for.

If your tables are intended to be a description of the sample data set, then no weighting is needed (or even appropriate). But if your tables are intended to be estimates of the distributions of the variables in the population your data are intended to generalize to, then you must apply weights.

The tables are for the analysis, and are to be generalised. I think this general "rule" makes sense. Thank you!

[Anders Astrup]

I would also say you may want to consider survey setting the data. The weights are likely a combination of weights to adjust for non-response bias and sampling probabilities. Analytics weights may not be appropriate. There should also be information on which method should be used to estimate the standard errors.

I did initially look into the -svy- settings, but since most of my analysis make use of the -pv- ado to cope with plausible values, it wasn't really an option. But you are right, there might be other issues.

[wbuchanan]

You should use the -mi- prefix commands. Plausible values are essentially draws from a posterior when all of the data are not observed and individual inference is not needed (e.g., when using spiral matrix booklet sampling for items). You can use svyset data with multiple imputation prefixes. That solves both problems for you. You’ll get the appropriate standard errors related to sampling error and on the measurement side, the plausible values will be combined using Rubin’s rule - regarding the decomposition of within and between imputation variances.