Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Parallelize loop

    I have a big loop over years and months in each year and then doing some computations in each month and appending them to a monthly panel.

    Is it possible to speed this up using parallel from ssc? And worth doing so for MP users?

  • #2
    You mean this user contributed command? I haven't used it myself, but I've done some parallel computing work on several occasions. The big question here is "does order matter?" Basically, if the calculation in the current iteration of the loop depends on the calculation made during the previous iteration, then it does not make sense to parallelize. Otherwise,

    Is it possible to speed this up using parallel from ssc?
    Yes, it should be possible, but not guaranteed. It depends on the size of the problem. In some cases, parallelization might actually slow things down. There is some overhead associated with spawning multiple threads/processes.

    And worth doing so for MP users?
    Possibly. It looks like the command starts several instances of Stata (headless) to process batch jobs in parallel. I imagine that could involve a lot of overhead on startup. On the other hand, It's certainly possible (even likely) that your for loop isn't automatically parallelized by Stata MP, in which case parallelizing explicitly yourself might speed things up.

    Comment

    Working...
    X