Bootstrap does not work with cmmixlogit -- error: insufficient obs... r(2000)

[Jeremy van Dijk]

Dear all,
I've been reading over all the previous posts I can find on the issue of -bootstrap- giving the error "insufficient observations to compute bootstrap standard errors", or "posting missing values", however none of the responses are helping fix my issue. So I am turning to you in the hope of finding some solution!

Firstly, as a general question, should -bootstrap- work with -cmmixlogit- or not? When I run it with the bootstrap option, it works, however, then when I do the same thing with the -bootstrap- command/prefix, it doesn't. So what could the issue be with the second version? Is it my code or a deeper issue?

Further basic testing of the bootstrap prefix on the same/similar model shows me that using -cmclogit- and (of course) no random vars does work, while -cmmixlogit- with no random vars doesn't (both work fine without -bootstrap-). So it seems to me like there's really an issue with -bootstrap : cmcmixlogit...-.

I post the code below, as well as a data sample extract. BTW, this is a much simpler version of what I actually want to do (hence why I need the prefix, not just the bootstrap option), but I hope an answer to this could suffice for my more complicated analysis too. I've also tried many different options and variants of bootstrapping, with no success.
Briefly about my data, it is the responses from stated preference choice tables from about 900 respondents. There are 6 alternatives per respondent of which there is 1 with 'chosen==1' and 5 'chosen==0'.

Code:

version 16.1 
*BOOTSTRAP OPTION WORKS
cmmixlogit chosen ib4.categ speed100 price10000 , rand(c.type1#c.marg_cst ) intmeth(halton) intpoint(20) nocons difficult vce(bootstrap)
*VERSUS: BOOTSTRAP PREFIX FAILS
bootstrap , cluster(id) idcluster(newid) nodrop seed(5) :  ///
cmmixlogit chosen ib4.categ speed100 price10000 , rand(c.type1#c.marg_cst ) intmeth(halton) intpoint(20) nocons difficult vce(cl id)

*Bootstrap cmclogit WORKS:
bootstrap , cluster(id) idcluster(newid)  nodrop seed(5) :    ///
    cmclogit chosen ib4.categ speed100 price10000 c.type1#c.marg_cst , nocons
*BUT Bootstrap cmmixlogit FAILS:
bootstrap , cluster(id) idcluster(newid) nodrop seed(5) : ///
    cmmixlogit chosen ib4.categ speed100 price10000 c.type1#c.marg_cst, nocons vce(cl id)

This is an extract of 15 respondents (out of 900 total). Unfortunately the mixed logit models above don't work on this subsample only - too few observations/cases. I am happy to supply a larger dataset if someone is willing and able to help with it.

* Example generated by -dataex-. For more info, type help dataex
clear
input int id float(chosen categ speed100 price10000 type1 marg_cst)
2004 0 1 1.35 2.9 0   2.6
2004 0 1  1.6   4 0   2.1
2004 0 2  1.6 3.3 0   3.3
2004 0 2  1.8 4.2 0   4.3
2004 1 3 1.68 2.3 1   5.8
2004 0 4 1.68 1.5 1     7
6447 0 1  1.5 5.3 0     3
6447 0 1    2 6.5 0   2.4
6447 0 2  1.9 5.1 0   4.3
6447 0 2  2.5   6 0   5.2
6447 0 3 2.15 4.4 1     7
6447 1 4 2.22 3.4 1     9
6761 0 1  1.4 3.5 0   2.8
6761 0 1  1.5 4.7 0   2.3
6761 1 2  1.7 3.7 0   3.5
6761 0 2  2.2 4.8 0   4.5
6761 0 3  1.8 3.3 1     6
6761 0 4 1.75 2.4 1   8.3
6861 0 1 1.35 2.9 0   2.6
6861 0 1  1.6   4 0   2.1
6861 0 2  1.6 3.3 0   3.3
6861 0 2  1.8 4.2 0   4.3
6861 0 3 1.68 2.3 1   5.8
6861 1 4 1.68 1.5 1     7
1973 0 1  1.5 5.3 0     3
1973 0 1    2 6.5 0   2.4
1973 0 2  1.9 5.1 0   4.3
1973 0 2  2.5   6 0   5.2
1973 0 3 2.15 4.4 1     7
1973 1 4 2.22 3.4 1     9
5176 0 1 1.35 2.9 0   2.6
5176 0 1  1.6   4 0   2.1
5176 0 2  1.6 3.3 0   3.3
5176 0 2  1.8 4.2 0   4.3
5176 0 3 1.68 2.3 1   5.8
5176 1 4 1.68 1.5 1     7
1226 0 1 1.35 2.9 0   2.6
1226 1 1  1.6   4 0   2.1
1226 0 2  1.6 3.3 0   3.3
1226 0 2  1.8 4.2 0   4.3
1226 0 3 1.68 2.3 1   5.8
1226 0 4 1.68 1.5 1     7
 508 0 1  1.4 3.5 0   2.8
 508 0 1  1.5 4.7 0   2.3
 508 0 2  1.7 3.7 0   3.5
 508 0 2  2.2 4.8 0   4.5
 508 0 3  1.8 3.3 1     6
 508 1 4 1.75 2.4 1   8.3
3302 0 1  1.5 5.3 0     3
3302 0 1    2 6.5 0   2.4
3302 0 2  1.9 5.1 0   4.3
3302 0 2  2.5   6 0   5.2
3302 0 3 2.15 4.4 1     7
3302 1 4 2.22 3.4 1     9
6350 1 1  1.8 6.8 0   4.1
6350 0 1  2.1 8.5 0   3.5
6350 0 2  1.7 6.5 0     5
6350 0 2  2.1 8.5 0 7.704
6350 0 3  1.9   7 1     8
6350 0 4  2.3 6.1 1    11
6188 0 1  1.4 3.5 0   2.8
6188 0 1  1.5 4.7 0   2.3
6188 0 2  1.7 3.7 0   3.5
6188 1 2  2.2 4.8 0   4.5
6188 0 3  1.8 3.3 1     6
6188 0 4 1.75 2.4 1   8.3
3867 0 1  1.8 6.8 0   4.1
3867 1 1  2.1 8.5 0   3.5
3867 0 2  1.7 6.5 0     5
3867 0 2  2.1 8.5 0 7.704
3867 0 3  1.9   7 1     8
3867 0 4  2.3 6.1 1    11
1741 0 1  1.4 3.5 0   2.8
1741 0 1  1.5 4.7 0   2.3
1741 0 2  1.7 3.7 0   3.5
1741 0 2  2.2 4.8 0   4.5
1741 0 3  1.8 3.3 1     6
1741 1 4 1.75 2.4 1   8.3
6864 0 1  1.5 5.3 0     3
6864 0 1    2 6.5 0   2.4
6864 1 2  1.9 5.1 0   4.3
6864 0 2  2.5   6 0   5.2
6864 0 3 2.15 4.4 1     7
6864 0 4 2.22 3.4 1     9
 628 0 1 1.35 2.9 0   2.6
 628 0 1  1.6   4 0   2.1
 628 1 2  1.6 3.3 0   3.3
 628 0 2  1.8 4.2 0   4.3
 628 0 3 1.68 2.3 1   5.8
 628 0 4 1.68 1.5 1     7
end

Thank you for any help you can provide!
Best regards,
Jeremy

[Jeremy van Dijk]

Follow-up: Sorry, I realised I missed one variable (alt) and a piece of code (-cmset-) as well in the above. Please see here the updated data extract and the code including the -cmset-.
(sorry if it's possible to edit the original post. I couldn't see how).

 
version 16.1 

cmset id alt  

*BOOTSTRAP OPTION WORKS
cmmixlogit chosen ib4.categ speed100 price10000 , rand(c.type1#c.marg_cst ) intmeth(halton) intpoint(20) nocons difficult vce(bootstrap)

*VERSUS: BOOTSTRAP PREFIX FAILS
bootstrap , cluster(id) idcluster(newid) nodrop seed(5) : ///
   cmmixlogit chosen ib4.categ speed100 price10000 , rand(c.type1#c.marg_cst ) intmeth(halton) intpoint(20) nocons difficult vce(cl id)

*Bootstrap cmclogit WORKS:
bootstrap , cluster(id) idcluster(newid)  nodrop seed(5) :    ///    
   cmclogit chosen ib4.categ speed100 price10000 c.type1#c.marg_cst , nocons

*BUT Bootstrap cmmixlogit FAILS:
bootstrap , cluster(id) idcluster(newid) nodrop seed(5) : ///
   cmmixlogit chosen ib4.categ speed100 price10000 c.type1#c.marg_cst, nocons vce(cl id)

* Example generated by -dataex-. For more info, type help dataex
clear
input int id byte alt float(chosen categ speed100 price10000 type1 marg_cst)
2004 1 0 1 1.35 2.9 0   2.6
2004 2 0 1  1.6   4 0   2.1
2004 3 0 2  1.6 3.3 0   3.3
2004 4 0 2  1.8 4.2 0   4.3
2004 5 1 3 1.68 2.3 1   5.8
2004 6 0 4 1.68 1.5 1     7
6447 1 0 1  1.5 5.3 0     3
6447 2 0 1    2 6.5 0   2.4
6447 3 0 2  1.9 5.1 0   4.3
6447 4 0 2  2.5   6 0   5.2
6447 5 0 3 2.15 4.4 1     7
6447 6 1 4 2.22 3.4 1     9
6761 1 0 1  1.4 3.5 0   2.8
6761 2 0 1  1.5 4.7 0   2.3
6761 3 1 2  1.7 3.7 0   3.5
6761 4 0 2  2.2 4.8 0   4.5
6761 5 0 3  1.8 3.3 1     6
6761 6 0 4 1.75 2.4 1   8.3
6861 1 0 1 1.35 2.9 0   2.6
6861 2 0 1  1.6   4 0   2.1
6861 3 0 2  1.6 3.3 0   3.3
6861 4 0 2  1.8 4.2 0   4.3
6861 5 0 3 1.68 2.3 1   5.8
6861 6 1 4 1.68 1.5 1     7
1973 1 0 1  1.5 5.3 0     3
1973 2 0 1    2 6.5 0   2.4
1973 3 0 2  1.9 5.1 0   4.3
1973 4 0 2  2.5   6 0   5.2
1973 5 0 3 2.15 4.4 1     7
1973 6 1 4 2.22 3.4 1     9
5176 1 0 1 1.35 2.9 0   2.6
5176 2 0 1  1.6   4 0   2.1
5176 3 0 2  1.6 3.3 0   3.3
5176 4 0 2  1.8 4.2 0   4.3
5176 5 0 3 1.68 2.3 1   5.8
5176 6 1 4 1.68 1.5 1     7
1226 1 0 1 1.35 2.9 0   2.6
1226 2 1 1  1.6   4 0   2.1
1226 3 0 2  1.6 3.3 0   3.3
1226 4 0 2  1.8 4.2 0   4.3
1226 5 0 3 1.68 2.3 1   5.8
1226 6 0 4 1.68 1.5 1     7
 508 1 0 1  1.4 3.5 0   2.8
 508 2 0 1  1.5 4.7 0   2.3
 508 3 0 2  1.7 3.7 0   3.5
 508 4 0 2  2.2 4.8 0   4.5
 508 5 0 3  1.8 3.3 1     6
 508 6 1 4 1.75 2.4 1   8.3
3302 1 0 1  1.5 5.3 0     3
3302 2 0 1    2 6.5 0   2.4
3302 3 0 2  1.9 5.1 0   4.3
3302 4 0 2  2.5   6 0   5.2
3302 5 0 3 2.15 4.4 1     7
3302 6 1 4 2.22 3.4 1     9
6350 1 1 1  1.8 6.8 0   4.1
6350 2 0 1  2.1 8.5 0   3.5
6350 3 0 2  1.7 6.5 0     5
6350 4 0 2  2.1 8.5 0 7.704
6350 5 0 3  1.9   7 1     8
6350 6 0 4  2.3 6.1 1    11
6188 1 0 1  1.4 3.5 0   2.8
6188 2 0 1  1.5 4.7 0   2.3
6188 3 0 2  1.7 3.7 0   3.5
6188 4 1 2  2.2 4.8 0   4.5
6188 5 0 3  1.8 3.3 1     6
6188 6 0 4 1.75 2.4 1   8.3
3867 1 0 1  1.8 6.8 0   4.1
3867 2 1 1  2.1 8.5 0   3.5
3867 3 0 2  1.7 6.5 0     5
3867 4 0 2  2.1 8.5 0 7.704
3867 5 0 3  1.9   7 1     8
3867 6 0 4  2.3 6.1 1    11
1741 1 0 1  1.4 3.5 0   2.8
1741 2 0 1  1.5 4.7 0   2.3
1741 3 0 2  1.7 3.7 0   3.5
1741 4 0 2  2.2 4.8 0   4.5
1741 5 0 3  1.8 3.3 1     6
1741 6 1 4 1.75 2.4 1   8.3
6864 1 0 1  1.5 5.3 0     3
6864 2 0 1    2 6.5 0   2.4
6864 3 1 2  1.9 5.1 0   4.3
6864 4 0 2  2.5   6 0   5.2
6864 5 0 3 2.15 4.4 1     7
6864 6 0 4 2.22 3.4 1     9
 628 1 0 1 1.35 2.9 0   2.6
 628 2 0 1  1.6   4 0   2.1
 628 3 1 2  1.6 3.3 0   3.3
 628 4 0 2  1.8 4.2 0   4.3
 628 5 0 3 1.68 2.3 1   5.8
 628 6 0 4 1.68 1.5 1     7
end

[William Lisowski]

Firstly, as a general question, should -bootstrap- work with -cmmixlogit- or not?

In the output of help cmmixlogit we see

bootstrap, by, collect, jackknife, statsby, and svy are allowed; see prefix.

so the answer to that question is yes, it should work.

This is beyond my area of expertise, but let me make this observation.

My understanding is that when bootstrapping, cmlogit creates newid as the cluster identifier in the bootstrapped data (since a given id may appear more than once in the bootstrap sample). Perhaps your vce(cl id) option should read vce(cl newid).

[Jeremy van Dijk]

Hil William,
Thanks for your response. I did see that note in the help file as well, and, as I said, the bootstrap vce option does work, so I'm not sure what the issue is with the prefix version.
I appreciate your suggestion, but it does not work and gives the following error message: option cluster() of bootstrap conflicts with option cluster() of cmmixlogit.
And running it without any vce option stated, it gives the original error again of insufficient observations.
I appreciate your attempt and hopefully someone else has a better idea of what's going on!
Best,

[Jeremy van Dijk]

So I have found a solution (or actually a workaround), in case anyone else encounters the same problem as me. I still do not understand why -bootstrap... : cmmixlogit- doesn't work, however the previous version of this command from Stata 15, -asmixlogit- does!

So:

version 16.1 
cmset id alt  
bootstrap , cluster(id) idcluster(newid) nodrop seed(5) : ///
   cmmixlogit chosen ib4.categ speed100 price10000 , rand(c.type1#c.marg_cst ) intmeth(halton) intpoint(50) nocons difficult 
* ^ doesn't work *
* but the following DOES! * 
bootstrap , cluster(id) idcluster(newid) nodrop seed(5) : ///
    asmixlogit chosen ib4.categ speed100 price10000 , rand(c.type1#c.marg_cst ) intmeth(halton) intpoint(50) nocons difficult  case(id) alt(alt) iterate(50)