Parallel code combined with forval loop

[Raphael Klei]

Hey all!

I have a do file which contains a forval loop with i from 1 to 250. This loop contains a certain analysis and the output is saved to a datafile named after the value (1,2,3....) before the next value is done. Now I found the parallel code which should enable parallel computing. I have quite a good pc and could use up to 32 cores at once. Unfortunately I dont know how to set up the parallel code so it does my do file for e.g. 32 values and once and will then continues with the other ones. Is this even possible? The problem is that the analysis for each value takes up to 10 minutes and would take forever if I can't manage to do it parallel.

Thanks a lot in advance and merry christmas to everyone!

Best regards
Raphael

[ericmelse]

Hi Raphael,

parallel is a community contributed command which indeed could meet your needs.
In The Stata Journal, 2019 19(3), 667-684, a paper about it was published by GeorgeVega Yon and Brian Quistorf: parallel: A command for parallel computing.

[Matthew Alexander]

Hi Raphael,
The paper that Eric identifies contains a thorough overview of parallel syntax - as do the help files for parallel.
That said, there are a fair number of different options - and so perhaps you are not entirely sure about which one you should use.
From what you describe above, I would advise you to use the program syntax. It would be something like this

parallel initialize 6
capture program drop myprog
program define myprog

#####  Insert program ####

parallel, program(myprog) : myprog

Two additional options may be needed if you encounter errors.
The first is to add the - force - option to the initialisation of parallel (this may be needed if your default permissions are restrictive in some way)

parallel initialize 6, force s("C:\Program Files\Stata17\StataBE-64.exe")

The second is that you may benefit from saving the output of your loop to a postfile.

tempname tempf
postfile `tempf' what you want to save using permanentfilename, replace

forvalues i/250 {
    
    preserve

    ### Insert loop syntax ###

    post `tempf' (what) (you) (want) (to) (save)

    restore
}

postclose `tempf'

All the best,
Matt