Given the algorithm, how to shorten the computation time of a Stata program?

[Ingrid Qiu]

This is a quite general question. I think to shorten the computation time, most people will improve their algorithms. But right now, I want to know if we don't change the algorithm, what else we can do to shorten the computation time. For example, I was told that some built-in Stata commands use C plug-in to speed up. I am wondering whether there are other kinds of plug-in? For example, C++ plug-in, Matlab plug-in, etc.. And it seems like that there isn't too many materials about how the C plug-in works. I am also wondering where I can find more learning materials about the pulg-in thing.

I am also thinking there may be some general principle that we should keep in mind that can make our codes work more efficiently.

Any ideas will be welcomed. Thanks!

[Jorge Eduardo Perez Perez]

I think this question should be in the Stata section.

These guides are very helpful to learn how to write faster code:

1. Stata for very large datasets: http://www.nber.org/stata/efficient/
2. Joe Canner's advice on Stata for large datasets: http://www.stata.com/meeting/new-orl...13-canner.pptx

[Ingrid Qiu]

Em, actually, I write a Stata command. But most of the codes are written in the Mata environment. So I would like to know how to write faster codes in Mata. But still thanks, I will go to the two websites to see whether I can get help there.

[Mike Lacy]

The best advice I have is to write as much of your code as you can as matrix operations, rather than with any looping. My rule of thumb is that doing this can speed run time by a factor of 10.

[Sergiy Radyakin]

Ingrid, there are C plugins, C++ plugins, PASCAL plugins, FORTRAN plugins, ASSEMBLER plugins. How does it help?
Without knowing the details of your algorithm it's hard to advise.

Mike has given you a good advise to use matrix operations to the most. This is in line with reliance on C code, since matrix operations are mostly implemented as built-in commands (written in C).

I would like to add to that advise, that at the very beginning of Stata programming, try to rely more on commands/programs implemented by someone else. They are going to be almost surely faster and more reliable than the code you are going to write in your first week of Stata programming. As you get more professional, you will find that it is easier for you to write your own code, than search for an existing solution, understand the details and limitations of it, or search for bugs in it.

Similarly a general advise is to use a profiler to identify which sections are accountable for most of the execution time, then focus on revising those sections.

[Hua Peng (StataCorp)]

Usually a good first step to optimize your ado or Mata code is to find which section of code spending most of the execution time. For that, use the Stata timer command or Mata timer*() functions to profile your code.

help timer

help mf_timer

[Ingrid Qiu]

The best advice I have is to write as much of your code as you can as matrix operations, rather than with any looping. My rule of thumb is that doing this can speed run time by a factor of 10.

Thank you Mike. I will go back to my code and check whether I am writing efficient codes.

[Ingrid Qiu]

Ingrid, there are C plugins, C++ plugins, PASCAL plugins, FORTRAN plugins, ASSEMBLER plugins. How does it help?
Without knowing the details of your algorithm it's hard to advise.

I am wondering whether the computation time can be shortened by using these plugins, given the same numbers of loops.

[Ingrid Qiu]

Usually a good first step to optimize your ado or Mata code is to find which section of code spending most of the execution time. For that, use the Stata timer command or Mata timer*() functions to profile your code.

help timer

help mf_timer

Thanks, Hua. I will add this into my Stata codes.