Bootstrapping time

[Eung Il Kim]

Hello

I drew a SEM with SEM builder in STATA v.16. After that, I run Bootstrapping 5,000 but it took about 2hours but still running with this word (running sem on estimation sample). I attached the code for SEM.

sem (Nv -> Zsl, ) (Nv -> ZNautonomy, ) (Nv -> ZNrelatedness, ) (Nv -> ZNcompetence, ) (Nv -> ZRJobinsecurity, ) (Nv -> ZR28, ) (Nv -> Z711236, ) (ZR651 -> Nv, ) (ZR653 -> Nv, ) (ZR652 -> Nv, ) (Zsl -> ZQ631, ) (Zsl -> ZQ632, ) (Zsl -> ZQ633, ) (Zsl -> Z711236, ) (ZNautonomy -> Z711236, ) (ZNautonomy -> ZNQ49C, ) (ZNautonomy -> ZNQ49E, ) (ZNautonomy -> ZNQ49I, ) (ZNautonomy -> ZNQ49N, ) (ZNrelatedness -> ZN52C, ) (ZNrelatedness -> Z711236, ) (ZNrelatedness -> ZN49A, ) (ZNrelatedness -> ZN49B, ) (ZNrelatedness -> ZN70D, ) (ZNcompetence -> Z711236, ) (ZNcompetence -> ZN70A, ) (ZNcompetence -> ZN70C, ) (ZNcompetence -> ZN70E, ) (ZRJobinsecurity -> Z711236, ) (ZR28 -> Z711236, ) (Z711236 -> ZR711, ) (Z711236 -> ZR712, ) (Z711236 -> ZR713, ) (Z711236 -> ZR716, ), vce(bootstrap, reps(5000) dots(1)) latent(Nv Zsl ZNautonomy ZNrelatedness ZNcompetence Z711236 ) nocapslatent

I like to hear some advice from you. Thank you

[Joro Kolev]

-sem- is a nonlinear iterative algorithm. When you bootstrap such finicky commands, you firstly try how fast it goes on say 10 or 30 bootstrap replication, and then you estimate the total time that it will take you to complete the desired amount of replications (5000 in your case).

[Felix Bittmann]

Since you have a dot displayed for every iteration it should be quite easy to estimate the total runtime after a few iterations as Joro suggests. If the runtime is too long you can either split it up and run the command on multiple machines and combine the results later on (use the saving options). Also have a look at parallel (https://github.com/gvegayon/parallel).

[Eung Il Kim]

Thank you Joro and Felix for good advice. I also followed Joro's advice and run 5 bootstrap replication but takes much time than expected. Just want to confirm that whether sem usually takes time to run bootstrapping.

Thank you again

[Joro Kolev]

I do not know how you measured it, but Stata has -[P] timer- and the user contributed -timeit-.

Everything that takes a lot time in itself, takes multiplicatively as much time to bootstrap. Say if one run of sem takes 10 seconds, (which you can check with -timeit-), 5000 bootstrap runs will take on average 5000 X 10 seconds.

Then there is the added problem for nonlinear iterative estimators like -sem- that for some configurations of the data (i.e., some bootstrap samples) the data might be ill conditioned, and the optimiser might get stuck in flat/non-concave regions.

[Carlo Lazzaro]

Eung:
as an aside to previous helpful comments, you may want to take a look in the literature of your research field and see how many bootstrap replications were used in articles/working-papers that cover your very same topic.

[Ishana Balan]

Hi,

I ran a longitudinal mediation model with multiple mediators using gsem. The program took a long time to run even for one iteration. So, as expected bootstrapping is taking a long time. I am running 200 iterations and the program has been running for 4 days now!!!! This was my command

bootstrap r(x) r(y) r(z) r(a),seed(12345) bca reps(200): boot1

The following is the output so far.

Jackknife replications (1,238)
----+--- 1 ---+--- 2 ---+--- 3 ---+--- 4 ---+--- 5
nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn 50
nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn 100
nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn 150
nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn 200
nnnnnnnnnnnnnnnnnnnn

If I am understanding correctly, I asked for 200 iterations which is over now? So, why have I not got the results yet? Why is the program still running?

I would really really like to switch off my laptop. Thanks in advance!