Hello,
I am using PISA data to carry out a pooled cross-country analysis of students academic achievement. PISA datasets contain a set of weights to calculate point statistics such that each country contributes proportionally to the number of students enrolled in its schools. The issue with this approach is that more populous countries will contribute more to the results. Thus, for pooled cross-country analyses, the weights need to be rescaled such that each country contributes in equal proportions to summary statistics. The PISA statistical manual (unfortunately it only reports guidelines for analyses in SPSS or SAS; see link below, p. 167) recommends the following computations to rescale the weight, if no data are missing (see I developed the STATA code):
1. Calculate the sum of the final student weight ( w_fstuwt) within each group defined by the "cnt" variable: egen sum_w_fstuwt = total(w_fstuwt), by(cnt)
2. Scale the final student weight based on the sum of weights calculated in step 1 (sum_w_fstuwt), and multiplying by 1000 (or 5000 for PISA 2015 and 2018): gen adjusted_w_fstuwt = (w_fstuwt / sum_w_fstuwt) * 1000
3. Repeat Step 2 computation for variables "w_fstr1" through "w_fstr80" and scale each of these variables by dividing by the sum of weights variable (sum_w_fstuwt) and multiplying by 1000.
forval i = 1/80 {
gen w_fstr_adjusted`i' = (w_fstr`i' / sum_w_fstuwt) * 1000
}
I can then use the rescaled weights to calculate point statistics. In the example below I am using the REPEST package to estimate the mean reading score for a pooled set of countries.
repest PISA, estimate(means pv@read) svyparms([NBpv(5) final_weight_name(adjusted_w_fstuwt) rep_weight_name(w_fstr_adjusted) NREP(80)])
However, I came across an article today that suggests a different computation formula. Gonzalez (2012; link below), recommends rescaling the weights are as follows:
1. Select the constant to which the sum of the weights will be rescaled (in our example, 1000).
2. Compute the sum of the sampling weights within each of the groups (in this case, countries) within which the rescaling will be done.
3. Multiply the sample weights by the result of dividing the constant selected in Step1 above by the sum of the weights in Step 2.
The second issue I have is that the PISA data sets contain "senate weights". These weights sum up to the same constant value within each country, and thus can be used to weight all countries equally. However, the PISA data analysis manual does not utilize senate weights in the rescaling computation. It is not clear to me why this is and how the senate weight variable is used.
Can anyone speak to the correct method of rescaling the weights provide in PISA using STATA?
PISA Data Analysis Manual (SPSS)
https://www.oecd-ilibrary.org/docser...ry%20estimates.
PISA Data Analysis Manual (SAS)
https://www.oecd-ilibrary.org/educat...89264056251-en
Gonzalez (2012)
https://ierinstitute.org/fileadmin/D...ical_Notes.pdf
I am using PISA data to carry out a pooled cross-country analysis of students academic achievement. PISA datasets contain a set of weights to calculate point statistics such that each country contributes proportionally to the number of students enrolled in its schools. The issue with this approach is that more populous countries will contribute more to the results. Thus, for pooled cross-country analyses, the weights need to be rescaled such that each country contributes in equal proportions to summary statistics. The PISA statistical manual (unfortunately it only reports guidelines for analyses in SPSS or SAS; see link below, p. 167) recommends the following computations to rescale the weight, if no data are missing (see I developed the STATA code):
1. Calculate the sum of the final student weight ( w_fstuwt) within each group defined by the "cnt" variable: egen sum_w_fstuwt = total(w_fstuwt), by(cnt)
2. Scale the final student weight based on the sum of weights calculated in step 1 (sum_w_fstuwt), and multiplying by 1000 (or 5000 for PISA 2015 and 2018): gen adjusted_w_fstuwt = (w_fstuwt / sum_w_fstuwt) * 1000
3. Repeat Step 2 computation for variables "w_fstr1" through "w_fstr80" and scale each of these variables by dividing by the sum of weights variable (sum_w_fstuwt) and multiplying by 1000.
forval i = 1/80 {
gen w_fstr_adjusted`i' = (w_fstr`i' / sum_w_fstuwt) * 1000
}
I can then use the rescaled weights to calculate point statistics. In the example below I am using the REPEST package to estimate the mean reading score for a pooled set of countries.
repest PISA, estimate(means pv@read) svyparms([NBpv(5) final_weight_name(adjusted_w_fstuwt) rep_weight_name(w_fstr_adjusted) NREP(80)])
However, I came across an article today that suggests a different computation formula. Gonzalez (2012; link below), recommends rescaling the weights are as follows:
1. Select the constant to which the sum of the weights will be rescaled (in our example, 1000).
2. Compute the sum of the sampling weights within each of the groups (in this case, countries) within which the rescaling will be done.
3. Multiply the sample weights by the result of dividing the constant selected in Step1 above by the sum of the weights in Step 2.
The second issue I have is that the PISA data sets contain "senate weights". These weights sum up to the same constant value within each country, and thus can be used to weight all countries equally. However, the PISA data analysis manual does not utilize senate weights in the rescaling computation. It is not clear to me why this is and how the senate weight variable is used.
Can anyone speak to the correct method of rescaling the weights provide in PISA using STATA?
PISA Data Analysis Manual (SPSS)
https://www.oecd-ilibrary.org/docser...ry%20estimates.
PISA Data Analysis Manual (SAS)
https://www.oecd-ilibrary.org/educat...89264056251-en
Gonzalez (2012)
https://ierinstitute.org/fileadmin/D...ical_Notes.pdf
