How to put a mata ml evaluator in a SSC package

[Cyrus Levy]

I have two files: File1 (ado) for an estimation command, and File2 (mata) for the mata based ml estimation function. File2 has mata mlib lines to create a library, and when I compile File2 on my computer, File1 works well. File1 would only work if File2 is run at least once by the user.

I want to make this program available to other users on SSC, and I know about how to upload ado files on SSC, but I am not sure how to upload an ado file on SSC that relies on the compilation of a mata file. How is it generally done? Do you put those two files in a folder and a readme text file to tell the user to compile the mata file first? Or, when you install a package from SSC, does it automatically compile the mata file for the user so that the ado file would work without a need to manually compile the mata file first? Or do you do something in File1 so that File1 would compile File2 when File1 is run by the user?

Below is a simplified example:

// File1 (ado)
...
program Estimate, eclass
...
ml model lf myprob() (...)
ml max
...
end


// File2 (mata)
mata:
void myprob(transmorphic scalar ML, real rowvector b, real colvector lnfj) {
    depvar = moptimize_util_depvar(ML, 1)
    xb = moptimize_util_xb(ML,b,1)
    lnfj = ...
}
end
mata: mata mlib create lmyprob, dir(PLUS) replace
mata: mata mlib add lmyprob myprob(), dir(PLUS)

[Sven-Kristjan Bormann]

Why do want to not pre-compile the evaluator and ship the compiled mlib?
You could also add in the beginning of your code a check for the existence of the mlib-file and compile it during the first call of your evaluator. Something like

// File1 (ado)
...
program Estimate, eclass
...
capture findfile lmyprob.mlib
if !rc do File2.mata
ml model lf myprob() (...)
ml max
...
end

[Maarten Buis]

As Sven Kristjan mentioned, you can add smaller Mata routines at the bottom of an .ado file. If the Mata routines become larger, then I prefer to add the compiled Mata library together with the source. The compiled library makes sure that everybody can run the program directly, just as any other Stata program. The source-file if for those who don't like pre-compiled programs from the internet, and for those who want to program similar things and wonder how I did things.

[Alvaro Gutierrez]

Regarding this topic. I am solving a similar problem where I already have written a couple of likelihood using

ml

but relying strongly on MATA calculations for my likelihoods because they are quite complex and involve a panel data, hence the log-likelihood contribution cannot be calculated separately for each observation.

This being said, I am using a MATA program to make the calculations faster than simply using ado based calculations. So far my code looks like the following one:

mata:
    void my_command(transmorphic scalar M, real scalar todo,
    real rowvector b, real scalar lnf,
    real rowvector g, real matrix H)
 {
    
    real matrix panvar  
    real matrix paninfo 
    real scalar npanels 

    Y = moptimize_util_depvar(M, 1)   
    X= moptimize_init_eq_indepvars(M,1)   

    // Create view of panels and set up panel-level processing
    st_view(panvar = ., ., st_global("MY_panel"))
    paninfo = panelsetup(panvar, 1)     
    npanels = panelstats(paninfo)[1] 
    lnfj = J(npanels, 1, 0) //  to storage the individuals controbutions to LL

    // loop over individuals (panels) 
for(n=1; n <= npanels; ++n) {

  ----> long calculations of individual contributions to the log-likelihood.....

        individual_contribution_to_LL= "some expression"

        lnfj[n] = ln(individual_contribution_to_LL)
         }
    //Summation over individuals (n)     
    lnf = moptimize_util_sum(M, lnfj)
}
end

global MY_panel id_ 
sort $MY_panel

ml model d0 my_command() ( Y = x1 x2 x3 , nocons)

Here is my question. Lately, I have read a post on the STATA-BLOG that suggests using same technique that you guys suggested to Cyrus Levy, but I was wrapping my code into an ado-file without creating a MATA compiled library because basically I didn't know about their existence. Therefore, my ado-file that wrap the code listed above is actually running using ml and just calling the MATA function that it is inside my ado-file, BUT...do you know if there are some guidelines that maybe are STRONGLY recommended for new STATA commands that aim to be published in Stata Journal about the style of the ado-files?

I am asking because I just finished the debugging process of my programs a couple of days ago and I am in the best moment to make changes because probably in a couple of weeks or months I will need to recap a lot to be able to add new features, options, or style modifications.

Finally, do you recommend me to ask this same question regarding the style of ado-files for new STATA commands in the general part?


PS: I have been following closely the post in the STATA-BLOG about writing adofiles and the book Maximum Likelihood Estimation with Stata, four Edition to write my programs.



Best regards,
Álvaro

[Sven-Kristjan Bormann]

You could post the same also in the general Stata forum. You should get there easier an answer then in this forum.
From what I have see so far is that you can continue like you already have. If you look at many popular, published commands (like oaxaca from Ben Jann for example) then you will see that they all have their Mata code after the Stata code. But they don't use a separate mlib-file.
Whether it is a matter of style or a matter functionality, I cannot tell. But my guess is that it depends on whether you expect to reuse your code somewhere else.