Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • weights with cmclogit

    Hi All,

    I am running choice models in STATA 16 MP with the new "cmclogit" command and ran into a few issues with the weights.

    The set up: I have a relatively large sample: 5,275,560 observations nested in 186,421 cases (between 14-34 alternative per observation). I run the same model once without weights, and once with weighted (using probability weights):


    //set up
    cmset cmset id_n major_name

    //unweighted model
    cmclogit choice $m1 if a==0

    //weighted model
    cmclogit choice $m1 if a==0 [pw=weight]

    The problem: the N in my weighted model dropped to 251,524 observations nested in 17966 cases. The reason for that, I found out it, is that it ran only for the subset of cases that had only 14 alternatives, even though all cases have weights, and the weights do not vary within cases. Any idea why it does that, and how can I fix it?

    Also note that this problem did not occur when I estimated the same model with the acslogit command.

    Thank you!

  • #2
    Hi Dafna,

    Using a simulated dataset with unbalanced alternatives, I am not able to reproduce the problem you describe. Below you can see results from cmclogit with and without weights as well as results from asclogit with weights, and as you can see the sample size is the same across all of them. If you can reproduce the problem on your end using a subsample of your data, please send the data and the exact code you are using to [email protected] so we can have a closer look at it.

    Joerg

    Code:
    . * Toy data:
    . clear
    
    . set seed 123
    
    . set obs 10000
    number of observations (_N) was 0, now 10,000
    
    . generate id = _n
    
    . local k 25
    
    . gen uexp = runiformint(10,`k')
    
    . expand uexp
    (165,134 observations created)
    
    . bysort id : gen alt = _n
    
    . gen double x1 = .
    (175,134 missing values generated)
    
    . gen double x2 = .
    (175,134 missing values generated)
    
    . gen double U  = .
    (175,134 missing values generated)
    
    . forval i = 1/`k' {
      2.         qui replace x1       = rnormal(0,1) if alt == `i'
      3.         qui replace x2       = rnormal(0,2) if alt == `i'
      4.         qui gen double xb`i' = 0.5*`i' + 3*x1 + 2*x2 if alt == `i'
      5.         qui gen double u`i'  = runiform()
      6.         qui gen double e`i'  = -log(-log(u`i'))
      7.         qui gen double U`i'  = xb`i' + e`i'     
      8.         qui replace U        = U`i' if !mi(U`i')
      9. }
    
    . gen y = .
    (175,134 missing values generated)
    
    . sort id alt
    
    . by id : egen double U_max = max(U)
    
    . by id : replace y = alt if U_max == U
    (10000 real changes made)
    
    . by id : gen first = _n == 1
    
    . gen wt = 1/runiform() if first
    (165,134 missing values generated)
    
    . by id : replace wt = wt[1]
    (165134 real changes made)
    
    . gen byte choice = alt == y
    
    . drop e* u* xb* U? U_max
    
    . keep  id alt choice x1 x2 wt
    
    . order id alt choice x1 x2 wt
    
    . 
    . 
    . * -cmset- data:
    . cmset id alt
    note: alternatives are unbalanced across choice sets; choice sets of different sizes found
    
                caseid variable:  id
          alternatives variable:  alt
    
    . 
    . * Models with and without weights:
    . cmclogit choice x1 x2
    
    Iteration 0:   log likelihood = -8428.4974  
    Iteration 1:   log likelihood =  -6138.707  
    Iteration 2:   log likelihood = -5957.4626  
    Iteration 3:   log likelihood = -5955.3261  
    Iteration 4:   log likelihood = -5955.3257  
    
    Conditional logit choice model                 Number of obs      =    175,134
    Case ID variable: id                           Number of cases    =      10000
    
    Alternatives variable: alt                     Alts per case: min =         10
                                                                  avg =       17.5
                                                                  max =         25
    
                                                      Wald chi2(2)    =    6409.72
    Log likelihood = -5955.3257                       Prob > chi2     =     0.0000
    
    ------------------------------------------------------------------------------
          choice |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
    -------------+----------------------------------------------------------------
    alt          |
              x1 |    3.05208   .0406792    75.03   0.000      2.97235     3.13181
              x2 |   2.020762   .0258394    78.20   0.000     1.970117    2.071406
    -------------+----------------------------------------------------------------
    1            |  (base alternative)
    -------------+----------------------------------------------------------------
    2            |
           _cons |   .6462814   .1962712     3.29   0.001     .2615968    1.030966
    -------------+----------------------------------------------------------------
    3            |
           _cons |   1.099376   .1859789     5.91   0.000     .7348639    1.463888
    -------------+----------------------------------------------------------------
    4            |
           _cons |   1.388963   .1819275     7.63   0.000     1.032391    1.745534
    -------------+----------------------------------------------------------------
    5            |
           _cons |    2.07745   .1744805    11.91   0.000     1.735474    2.419425
    -------------+----------------------------------------------------------------
    6            |
           _cons |    2.74877   .1699281    16.18   0.000     2.415718    3.081823
    -------------+----------------------------------------------------------------
    7            |
           _cons |   3.092922   .1697148    18.22   0.000     2.760287    3.425557
    -------------+----------------------------------------------------------------
    8            |
           _cons |   3.687629   .1689682    21.82   0.000     3.356458    4.018801
    -------------+----------------------------------------------------------------
    9            |
           _cons |   4.076979   .1681634    24.24   0.000     3.747384    4.406573
    -------------+----------------------------------------------------------------
    10           |
           _cons |   4.543436   .1669247    27.22   0.000      4.21627    4.870602
    -------------+----------------------------------------------------------------
    11           |
           _cons |    5.08789   .1705131    29.84   0.000     4.753691     5.42209
    -------------+----------------------------------------------------------------
    12           |
           _cons |   5.612841   .1733053    32.39   0.000     5.273169    5.952513
    -------------+----------------------------------------------------------------
    13           |
           _cons |   6.059629    .177851    34.07   0.000     5.711047     6.40821
    -------------+----------------------------------------------------------------
    14           |
           _cons |   6.541667   .1802193    36.30   0.000     6.188444    6.894891
    -------------+----------------------------------------------------------------
    15           |
           _cons |   7.094249   .1838862    38.58   0.000     6.733839     7.45466
    -------------+----------------------------------------------------------------
    16           |
           _cons |    7.61142   .1874264    40.61   0.000     7.244071    7.978769
    -------------+----------------------------------------------------------------
    17           |
           _cons |   8.184652   .1929816    42.41   0.000     7.806415    8.562889
    -------------+----------------------------------------------------------------
    18           |
           _cons |   8.593678   .1954606    43.97   0.000     8.210582    8.976774
    -------------+----------------------------------------------------------------
    19           |
           _cons |   9.007631   .2004975    44.93   0.000     8.614663    9.400598
    -------------+----------------------------------------------------------------
    20           |
           _cons |    9.57946   .2065485    46.38   0.000     9.174633    9.984288
    -------------+----------------------------------------------------------------
    21           |
           _cons |   10.08145   .2120842    47.54   0.000      9.66577    10.49713
    -------------+----------------------------------------------------------------
    22           |
           _cons |   10.64216   .2209306    48.17   0.000     10.20914    11.07517
    -------------+----------------------------------------------------------------
    23           |
           _cons |   11.13749   .2297632    48.47   0.000     10.68716    11.58782
    -------------+----------------------------------------------------------------
    24           |
           _cons |    11.6989   .2470376    47.36   0.000     11.21472    12.18309
    -------------+----------------------------------------------------------------
    25           |
           _cons |   12.24825   .2833676    43.22   0.000     11.69286    12.80364
    ------------------------------------------------------------------------------
    
    . est store m1
    
    . 
    . cmclogit choice x1 x2 [pw=wt]
    
    Iteration 0:   log pseudolikelihood =  -64895.31  
    Iteration 1:   log pseudolikelihood = -53648.076  
    Iteration 2:   log pseudolikelihood =   -43672.2  
    Iteration 3:   log pseudolikelihood = -43548.539  
    Iteration 4:   log pseudolikelihood = -43548.034  
    Iteration 5:   log pseudolikelihood = -43548.034  
    
    Conditional logit choice model                 Number of obs      =    175,134
    Case ID variable: id                           Number of cases    =      10000
    
    Alternatives variable: alt                     Alts per case: min =         10
                                                                  avg =       17.5
                                                                  max =         25
    
                                                      Wald chi2(2)    =     363.80
    Log pseudolikelihood = -43548.034                 Prob > chi2     =     0.0000
    
                                         (Std. Err. adjusted for clustering on id)
    ------------------------------------------------------------------------------
                 |               Robust
          choice |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
    -------------+----------------------------------------------------------------
    alt          |
              x1 |   3.451738   .2106132    16.39   0.000     3.038944    3.864532
              x2 |   2.184781   .1146772    19.05   0.000     1.960018    2.409544
    -------------+----------------------------------------------------------------
    1            |  (base alternative)
    -------------+----------------------------------------------------------------
    2            |
           _cons |   1.291761   .3814116     3.39   0.001      .544208    2.039314
    -------------+----------------------------------------------------------------
    3            |
           _cons |   1.807726   .3859883     4.68   0.000     1.051203    2.564249
    -------------+----------------------------------------------------------------
    4            |
           _cons |   1.825384   .6016592     3.03   0.002     .6461532    3.004614
    -------------+----------------------------------------------------------------
    5            |
           _cons |   2.627955   .7310866     3.59   0.000     1.195052    4.060858
    -------------+----------------------------------------------------------------
    6            |
           _cons |   3.363305   .4479759     7.51   0.000     2.485289    4.241322
    -------------+----------------------------------------------------------------
    7            |
           _cons |   4.372481   .5429639     8.05   0.000     3.308291     5.43667
    -------------+----------------------------------------------------------------
    8            |
           _cons |   3.741829   .4001458     9.35   0.000     2.957558    4.526101
    -------------+----------------------------------------------------------------
    9            |
           _cons |   4.704098   .4200532    11.20   0.000     3.880809    5.527388
    -------------+----------------------------------------------------------------
    10           |
           _cons |   5.336326   .4752438    11.23   0.000     4.404865    6.267787
    -------------+----------------------------------------------------------------
    11           |
           _cons |   6.208026   .5314768    11.68   0.000     5.166351    7.249702
    -------------+----------------------------------------------------------------
    12           |
           _cons |   6.651438   .4576069    14.54   0.000     5.754545    7.548331
    -------------+----------------------------------------------------------------
    13           |
           _cons |   5.936294   .4737851    12.53   0.000     5.007692    6.864895
    -------------+----------------------------------------------------------------
    14           |
           _cons |   7.408028   .4975147    14.89   0.000     6.432917    8.383139
    -------------+----------------------------------------------------------------
    15           |
           _cons |   8.374314   .5347899    15.66   0.000     7.326146    9.422483
    -------------+----------------------------------------------------------------
    16           |
           _cons |   8.661226   .6072163    14.26   0.000     7.471104    9.851348
    -------------+----------------------------------------------------------------
    17           |
           _cons |   9.754176   .6412963    15.21   0.000     8.497258    11.01109
    -------------+----------------------------------------------------------------
    18           |
           _cons |   10.10025   .6136957    16.46   0.000     8.897424    11.30307
    -------------+----------------------------------------------------------------
    19           |
           _cons |   10.28346   .7061948    14.56   0.000     8.899343    11.66758
    -------------+----------------------------------------------------------------
    20           |
           _cons |   10.60398   .6868559    15.44   0.000     9.257763    11.95019
    -------------+----------------------------------------------------------------
    21           |
           _cons |   11.64026   .7492649    15.54   0.000     10.17173     13.1088
    -------------+----------------------------------------------------------------
    22           |
           _cons |    12.3033   .9602244    12.81   0.000      10.4213    14.18531
    -------------+----------------------------------------------------------------
    23           |
           _cons |   13.38192   .8271476    16.18   0.000     11.76074    15.00309
    -------------+----------------------------------------------------------------
    24           |
           _cons |   14.76299    .887433    16.64   0.000     13.02366    16.50233
    -------------+----------------------------------------------------------------
    25           |
           _cons |   11.68168    .924442    12.64   0.000     9.869809    13.49355
    ------------------------------------------------------------------------------
    
    . est store m2
    
    . 
    . asclogit choice x1 x2 [pw=wt], case(id) alt(alt)
    
    Iteration 0:   log pseudolikelihood =  -64895.31  
    Iteration 1:   log pseudolikelihood = -53648.076  
    Iteration 2:   log pseudolikelihood =   -43672.2  
    Iteration 3:   log pseudolikelihood = -43548.539  
    Iteration 4:   log pseudolikelihood = -43548.034  
    Iteration 5:   log pseudolikelihood = -43548.034  
    
    Alternative-specific conditional logit         Number of obs      =    175,134
    Case ID variable: id                           Number of cases    =      10000
    
    Alternatives variable: alt                     Alts per case: min =         10
                                                                  avg =       17.5
                                                                  max =         25
    
                                                      Wald chi2(2)    =     363.80
    Log pseudolikelihood = -43548.034                 Prob > chi2     =     0.0000
    
                                         (Std. Err. adjusted for clustering on id)
    ------------------------------------------------------------------------------
                 |               Robust
          choice |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
    -------------+----------------------------------------------------------------
    alt          |
              x1 |   3.451738   .2106132    16.39   0.000     3.038944    3.864532
              x2 |   2.184781   .1146772    19.05   0.000     1.960018    2.409544
    -------------+----------------------------------------------------------------
    1            |  (base alternative)
    -------------+----------------------------------------------------------------
    2            |
           _cons |   1.291761   .3814116     3.39   0.001      .544208    2.039314
    -------------+----------------------------------------------------------------
    3            |
           _cons |   1.807726   .3859883     4.68   0.000     1.051203    2.564249
    -------------+----------------------------------------------------------------
    4            |
           _cons |   1.825384   .6016592     3.03   0.002     .6461532    3.004614
    -------------+----------------------------------------------------------------
    5            |
           _cons |   2.627955   .7310866     3.59   0.000     1.195052    4.060858
    -------------+----------------------------------------------------------------
    6            |
           _cons |   3.363305   .4479759     7.51   0.000     2.485289    4.241322
    -------------+----------------------------------------------------------------
    7            |
           _cons |   4.372481   .5429639     8.05   0.000     3.308291     5.43667
    -------------+----------------------------------------------------------------
    8            |
           _cons |   3.741829   .4001458     9.35   0.000     2.957558    4.526101
    -------------+----------------------------------------------------------------
    9            |
           _cons |   4.704098   .4200532    11.20   0.000     3.880809    5.527388
    -------------+----------------------------------------------------------------
    10           |
           _cons |   5.336326   .4752438    11.23   0.000     4.404865    6.267787
    -------------+----------------------------------------------------------------
    11           |
           _cons |   6.208026   .5314768    11.68   0.000     5.166351    7.249702
    -------------+----------------------------------------------------------------
    12           |
           _cons |   6.651438   .4576069    14.54   0.000     5.754545    7.548331
    -------------+----------------------------------------------------------------
    13           |
           _cons |   5.936294   .4737851    12.53   0.000     5.007692    6.864895
    -------------+----------------------------------------------------------------
    14           |
           _cons |   7.408028   .4975147    14.89   0.000     6.432917    8.383139
    -------------+----------------------------------------------------------------
    15           |
           _cons |   8.374314   .5347899    15.66   0.000     7.326146    9.422483
    -------------+----------------------------------------------------------------
    16           |
           _cons |   8.661226   .6072163    14.26   0.000     7.471104    9.851348
    -------------+----------------------------------------------------------------
    17           |
           _cons |   9.754176   .6412963    15.21   0.000     8.497258    11.01109
    -------------+----------------------------------------------------------------
    18           |
           _cons |   10.10025   .6136957    16.46   0.000     8.897424    11.30307
    -------------+----------------------------------------------------------------
    19           |
           _cons |   10.28346   .7061948    14.56   0.000     8.899343    11.66758
    -------------+----------------------------------------------------------------
    20           |
           _cons |   10.60398   .6868559    15.44   0.000     9.257763    11.95019
    -------------+----------------------------------------------------------------
    21           |
           _cons |   11.64026   .7492649    15.54   0.000     10.17173     13.1088
    -------------+----------------------------------------------------------------
    22           |
           _cons |    12.3033   .9602244    12.81   0.000      10.4213    14.18531
    -------------+----------------------------------------------------------------
    23           |
           _cons |   13.38192   .8271476    16.18   0.000     11.76074    15.00309
    -------------+----------------------------------------------------------------
    24           |
           _cons |   14.76299    .887433    16.64   0.000     13.02366    16.50233
    -------------+----------------------------------------------------------------
    25           |
           _cons |   11.68168    .924442    12.64   0.000     9.869809    13.49355
    ------------------------------------------------------------------------------
    
    . est store m3
    
    . 
    . * Results:
    . est table m1 m2 m3, se stats(N ll)
    
    -----------------------------------------------------
        Variable |     m1           m2           m3      
    -------------+---------------------------------------
    alt          |
              x1 |  3.0520798    3.4517379    3.4517379  
                 |  .04067918    .21061317    .21061317  
              x2 |  2.0207616     2.184781     2.184781  
                 |  .02583944    .11467722    .11467722  
    -------------+---------------------------------------
    2            |
           _cons |  .64628135    1.2917609    1.2917609  
                 |  .19627125    .38141157    .38141157  
    -------------+---------------------------------------
    3            |
           _cons |  1.0993759    1.8077262    1.8077262  
                 |  .18597892    .38598832    .38598832  
    -------------+---------------------------------------
    4            |
           _cons |  1.3889627    1.8253835    1.8253835  
                 |  .18192745    .60165919    .60165919  
    -------------+---------------------------------------
    5            |
           _cons |  2.0774498    2.6279551    2.6279551  
                 |  .17448052    .73108661    .73108661  
    -------------+---------------------------------------
    6            |
           _cons |  2.7487705    3.3633055    3.3633055  
                 |  .16992808    .44797588    .44797588  
    -------------+---------------------------------------
    7            |
           _cons |  3.0929218    4.3724805    4.3724805  
                 |  .16971479    .54296387    .54296387  
    -------------+---------------------------------------
    8            |
           _cons |  3.6876292    3.7418293    3.7418293  
                 |  .16896822    .40014584    .40014584  
    -------------+---------------------------------------
    9            |
           _cons |  4.0769785    4.7040985    4.7040985  
                 |  .16816339    .42005321    .42005321  
    -------------+---------------------------------------
    10           |
           _cons |  4.5434359    5.3363258    5.3363258  
                 |  .16692468    .47524382    .47524382  
    -------------+---------------------------------------
    11           |
           _cons |  5.0878905    6.2080264    6.2080264  
                 |  .17051308    .53147681    .53147681  
    -------------+---------------------------------------
    12           |
           _cons |  5.6128409    6.6514379    6.6514379  
                 |  .17330525    .45760693    .45760693  
    -------------+---------------------------------------
    13           |
           _cons |  6.0596286    5.9362936    5.9362936  
                 |  .17785102    .47378512    .47378512  
    -------------+---------------------------------------
    14           |
           _cons |  6.5416673    7.4080283    7.4080283  
                 |  .18021928    .49751469    .49751469  
    -------------+---------------------------------------
    15           |
           _cons |  7.0942493    8.3743144    8.3743144  
                 |  .18388623    .53478987    .53478987  
    -------------+---------------------------------------
    16           |
           _cons |  7.6114204     8.661226     8.661226  
                 |  .18742636    .60721626    .60721626  
    -------------+---------------------------------------
    17           |
           _cons |  8.1846518    9.7541757    9.7541757  
                 |  .19298164     .6412963     .6412963  
    -------------+---------------------------------------
    18           |
           _cons |  8.5936778    10.100245    10.100245  
                 |  .19546064    .61369566    .61369566  
    -------------+---------------------------------------
    19           |
           _cons |  9.0076307    10.283459    10.283459  
                 |  .20049746    .70619478    .70619478  
    -------------+---------------------------------------
    20           |
           _cons |  9.5794604    10.603976    10.603976  
                 |  .20654855    .68685587    .68685587  
    -------------+---------------------------------------
    21           |
           _cons |  10.081448    11.640265    11.640265  
                 |   .2120842    .74926493    .74926493  
    -------------+---------------------------------------
    22           |
           _cons |  10.642157    12.303302    12.303302  
                 |  .22093062    .96022439    .96022439  
    -------------+---------------------------------------
    23           |
           _cons |  11.137488    13.381915    13.381915  
                 |  .22976318    .82714759    .82714759  
    -------------+---------------------------------------
    24           |
           _cons |  11.698903    14.762994    14.762994  
                 |  .24703761    .88743304    .88743304  
    -------------+---------------------------------------
    25           |
           _cons |  12.248251    11.681682    11.681682  
                 |  .28336757    .92444202    .92444202  
    -------------+---------------------------------------
    Statistics   |                                       
               N |     175134       175134       175134  
              ll | -5955.3257   -43548.034   -43548.034  
    -----------------------------------------------------
                                             legend: b/se

    Comment

    Working...
    X