Use of repest for carrying out multilevel analysis with PIAAC

[Luis Ortiz]

Dear members of the list,

I am trying to run a multilevel analysis using PIAAC data.

Knowing the complex survey design of PIAAC, I enquired the OECD. An official of this organism recommended us considering the use of REPEST command in order to "estimate statistics using replicate weights (BRR weights, Jackknife replicate weights,...), thus accounting for complex survey designs in the estimation of sampling variances".

Following the syntax of 'repest', we run the following command, where 'univ' is our dichotomous dependent variable, and 'edufath', 'female' and 'age' are our independent variables.

repest PIAAC, estimate(stata: xtmelogit univ i.edufath female age  || cntryid3:)

We obtain the following output, recommending us to "debug the command within the estimate option", but it is hard for us to understand what this exactly means, and how to proceed.

. repest PIAAC, estimate(stata: xtmelogit univ i.edufath female age || cntryid3:)
(note: file C:\Users\U19030\AppData\Local\Temp\ST_02000005.tmp not found)
file C:\Users\U19030\AppData\Local\Temp\ST_02000005.tmp saved

_pooled. 
   There were no successful estimates for  = _pooled.
 Consider debugging the command within the estimate option:
 xtmelogit univ i.edufath female age || cntryid3:    [pw = spfwt0]  
weights not allowed

We do not know if the problem lies in one of the weights implicitly consider by the command 'repest' or rather in the fact that 'repest' does not allow the use of xtmelogit.

We would appreciate any guidance to solve this problem.

Thanks for our attention and kind regards

Luis Ortiz

[Luis Ortiz]

Dear all,

As I said in my previous message, I am trying to carry out a multilevel analysis of graduate and postgraduate educational attainment with data on 28 eight countries drawn from PIAAC survey.

This is a survey with a comple sample design, and this needs to be accounted for in the analysis. A OECD analyst recommended me to use a Stata command (repest) that has been especifically created for dealing with the complex sample design of a number of OECD databases; PIAAC, among them.

Yet, as I have argued in my previous message, this command does not seem to work very well with the commands that I have been using so far for my multilevel logistic regressions (xtlogit / xtmelogit). Therefore, I have tried to resort to more standard ways of dealing with the problem of the sample design of PIAAC: svy, svyset

Thus, I use first svyset to declare the survey design of the dataset, with its differents weights. I was informed of the need to include all of them. I have proceeded as recommended in p.6 of the the following document: PIAACTOOLS: Stata programs for data analysis with Stata There, you can see the following line

. svyset [pw=spfwt0], jkrweight(spfwt1-spfwt80) vce(jackknife) mse

      pweight: spfwt0
          VCE: jackknife
          MSE: on
    jkrweight: spfwt1 spfwt2 spfwt3 spfwt4 spfwt5 spfwt6 spfwt7 spfwt8
               spfwt9 spfwt10 spfwt11 spfwt12 spfwt13 spfwt14 spfwt15
               spfwt16 spfwt17 spfwt18 spfwt19 spfwt20 spfwt21 spfwt22
               spfwt23 spfwt24 spfwt25 spfwt26 spfwt27 spfwt28 spfwt29
               spfwt30 spfwt31 spfwt32 spfwt33 spfwt34 spfwt35 spfwt36
               spfwt37 spfwt38 spfwt39 spfwt40 spfwt41 spfwt42 spfwt43
               spfwt44 spfwt45 spfwt46 spfwt47 spfwt48 spfwt49 spfwt50
               spfwt51 spfwt52 spfwt53 spfwt54 spfwt55 spfwt56 spfwt57
               spfwt58 spfwt59 spfwt60 spfwt61 spfwt62 spfwt63 spfwt64
               spfwt65 spfwt66 spfwt67 spfwt68 spfwt69 spfwt70 spfwt71
               spfwt72 spfwt73 spfwt74 spfwt75 spfwt76 spfwt77 spfwt78
               spfwt79 spfwt80
  Single unit: missing
     Strata 1: <one>
         SU 1: <observations>
        FPC 1: <zero> 

Knowing that neither xtmelogit nor xtlogit support the svy prefix, I then try with melogit. But something wrong happens, and I am not able to decipher where exactly is the problem:

. svy: melogit univ i.edufath female age || cntryid3:  
melogit is not supported by svy with vce(jackknife); see help svy
estimation for a list of Stata estimation commands that are supported by
svy 

Is it a problem of the version of Stata that I am using (Stata 14)? Is it a problem of the way I have 'svysetted' my data?

Any clue about the problem would be very much appreciated

Thanks a lot for your attention

Luis Ortiz

[Letizia Gam]

Dear Luis,

I am experiencing the same issue with an analysis of similar survey data. I was wondering if you had solved the problem? if so, how?

Thanks very much,
Letizia Gam

[Beatrice Caniglia]

Dear professor Ortiz,

unfortunately I am facing the same issues working with PIAAC data and trying to perform a multilevel ordered logistic regression for the schooling effect on earnings across OECD countries. May I ask you if and how you overcame the above mentioned problems?

Thank you in advance for your support.
Beatrice Caniglia

[Giovanni Greco USERNAME]

Dear Professor Ortiz, and dear all,

I unfortunately do not have suggestions to your difficulties. In my case, basic logistic regressions with repest display different p-values depending on whether I have results in log odds or odds ratios. Of course, if I run the logistic regression without repest, p-values are exactly the same in the regression with log odds and in the regression with odds ratios. I fear there are a few things we are missing about the repest command that need to be clarified. And I would suggest to spread the word if a course on how to analyse PIAAC is made available.
Let's post in this chat if we have anything new concerning our doubts.

Best,
Giovanni Greco