When writing a large-scale programming project, do you use do files or ado files?

[Baptiste Ottino]

Hello everyone,

I'm writing a largish software project in Stata, that has to do with data preparation. I'm new to Stata programming, and I find myself constantly changing my mind between using do files and using ado files for the commands. On the one hand, do files seem more appropriate, since I'm adding a lot of new commands that I use only once or twice. It has the slight inconvenience that I have to run them in the beginning of every do file using them, but it gives me the possibility to group programs by category in the same do-file, as opposed to one program (and its submethods) per ado file.

On the other end, for distribution, I like that ado files can be installed in the PLUS sysdir, and that I can provide help files.

What is the standard practice for large Stata projects? Should I use ado files or do files? Or maybe a mix of both? Thank you very much!

edit: I should add, to be more precise, that most of these commands are "under-the-hood" commands, and that most if not all end users will never use them. Out of the 30+ commands, I think I will want 2-3 to be available to the end user.

[Amin Sofla]

You might want to try:

ssc install project
help project

[Nick Cox]

I'd give the same answer as for a small project. Do files are natural when you want to use specific variable names. It's possible to write ado files wiring in particular variable names, but by and large it's not my style and (wilder assertion) it's not especially good style. Ado files are for general tasks where the generality makes it worthwhile to abstract and to let the user specify variable names (for most tasks). Sometimes I have default variable names for output, but it's then good practice to allow user-specified alternatives. (If you can find exceptions in my public programs, either there's a good reason or I was just sloppy.)

I often see people trying to write a program when a do-file would be a better choice. Conversely, it's quite hard to grasp all the stuff that comes with a Stata program (strict sense) and a practical way to learn some programming can be through taking a do-file and generalising some of it.

Years ago (I'll change the details) I can remember a program where all variables of a certain kind were assumed to begin with t with further very specific conventions wired in and all this was undocumented in any help. I went !!! and was told "That's the convention in our group and we all agreed to follow it". In short. and with Stata pun intended, sometimes local conventions are just local conventions.

[Baptiste Ottino]

Thank you Amin and Nick for your answers.

Amin, -project- looks like a great tool, thanks. The only problem I see with it is the fact that it constantly -cd- into the directory of the current do file, which might be problem if I want to interact with user-specified relative paths. But I guess I could make sure to always use absolute paths (with -window fopen- for example).

From what I understand from both Nick's answer and -project-, projects are best written in terms of do files, unless the generalization really requires an ado file (for example, if the ado contains a complex method of regression over user-specified variables). So regardless of whether I end up using -project- or not, I should stick with do files.

I have a couple more questions. Nick, you wrote:

I often see people trying to write a program when a do-file would be a better choice.

Is it bad practice to write programs in do files? If you want a command you use several times in your project, but you don't want the end user to call directly for example?

Finally, can you write help files for do files? If not, how do you write documentation for do files outside of commenting?

Once more, thank you both!

[Nick Cox]

You can write a help file for anything you like. For example, you could collect jokes and put them in jokes.sthlp

Like anybody else, I have prejudices about style and don't always keep them to myself. But getting the code correct is usually more important than following anybody's style rules. At the same time, some style choices make getting correct (and checkably correct) code more difficult.

Some of the most puzzling code I see is written by people who evidently know some other software much better. That's not meant to be patronising. I came to Stata from a background of programming in various other languages and it took me quite a while to get used to Stata programming given that background. Nor is Stata an especially easy first programming language, not least because it's not intended to be a programming language strict sense!

[Amin Sofla]

I have prejudices about style and don't always keep them to myself.

Nick Cox : Could you please refer me to a page or a manual regarding the general guideline for the style in Stata? I think it would be very useful for the beginners like me.
Thanks in advance,

[Nick Cox]

I doubt that there is any single source here. StataCorp have their own house style but it has to be learned from their code. I wrote a piece on style in 2005 which I've not read for some while.


SJ-5-4 pr0018 . . . . . . . . . . . . Suggestions on Stata programming style
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . N. J. Cox
Q4/05 SJ 5(4):560--566 (no commands)
suggestions for good Stata programming style

[Amin Sofla]

Thank you very much Nick!

[Baptiste Ottino]

Some of the most puzzling code I see is written by people who evidently know some other software much better. That's not meant to be patronising. I came to Stata from a background of programming in various other languages and it took me quite a while to get used to Stata programming given that background.

That describes my experience perfectly. Stata programming is powerful, but you really need to get into the "Stata mindset". For this reason, the input of experienced Stata programmers is extremely valuable to me.

Thank you very much for your advice. Also, your article on Stata programming style is fantastic. It could be part of the official documentation.

[Richard Williams]

Nick's article on Stata programming style is available for free at

https://www.stata-journal.com/articl...article=pr0018

[Steve Samuels]

It is often useful to write a Stata program inside a do-file. That will be the case, for example when you want to bootstrap or simulate a series of commands. For examples, see the manual entry for simulate and Example 4 in the manual entry for bootstrap. There are other occasions when I've found use for a short program, for example here.

[Maarten Buis]

Is it bad practice to write programs in do files? If you want a command you use several times in your project, but you don't want the end user to call directly for example?

You don't have to put your entire data preparation and data analysis in one .do file. In fact that is in most cases bad practice as it will get just too long to be easily manageable and running it completely will take too long to be convenient. The latter is particularly problematic, as you want to make sure that your do file runs the same way every time you run it, and the results are not coming from some experimentation you did in between that did not make it in the do file. So it is important that you regularly run the file in its entirety. If that is inconvenient, then you don't do that as often as you should, and bugs will enter your code. So breaking up your do files into several shorter do files, is very important. When you do that, then it is a good idea to create a master .do file that will run the .do files in order (and comments to tell what each .do file does).

So that is a long winded way of saying that you can have .do files call other .do files, the command is

do dofilename

That is a possible alternative to writing a command for the repetitive commands. You can also pass local macros to that do file. If you type:

do dofilename foo bar

then at the beginning of dofilename the local macros `1' and `2' will contain "foo" and "bar" respectively. When that is the intended use of dofilename then I start that file with a comment stating that, so it is clear where those macros come from.

My general rule (which I don't always follow) is that I will write a new command for something that is going to be used in multiple projects. If it is just for one project I tend to work with .do files.