saving non-rectangular Mata objects

[Frank Erickson]

I am developing a program that will be run on a daily basis, but will use Mata objects that only change weekly. I would like to save and load these objects throughout the week without re-creating them (since it takes 3-4 hours to make them).

In the documentation and online, I have come across no way of saving Mata objects directly.

---

I could conceivably save them by exporting to Stata, but they are not rectangular so (i) I would need to figure out how (if at all) it is possible to convert my objects into tables; and (ii) converting to and from Stata format would likely be very inefficient.

When I say they are not rectangular, I mean...

• I have a column vector of pointers to vectors of row numbers. This describes a directed graph (starting from this row, which rows can I go to?).
• I have a matrix-valued function that I am evaluating on a grid of inputs and saving with asarray(fun,idx,mat)

In each of these two cases, I have a pretty clear idea on how I could make the data rectangular, (columns = row_number0, row_number1) and (columns = idx_cols, mat_cols), respectively. And I guess it's not so bad getting it back out by looping and selecting with [,] or select(,).

---

I'm just wondering if there is another way that I have missed. I can imagine facing messier data structures, and there certainly is some (albeit small) computational cost associated with the conversion in the case I'm facing now.

[Matthew J. Baker]

Frank -

It sounds like what you are after could be handled by a structured object in mata. This allows one to bind together an idiosyncratic bunch of objects, and is what is typically used to handle things like estimation results, where one has to save variable names, parameter estimates, standard errors, R-squared, etc.

As an example of how it works, suppose I have a couple of matrices, scalars, and a list of names that I want to save. I would go into Mata and create an object:

clear all
mata:
struct MyStruct {
    real matrix x,y
    real scalar  a,b,c
    string matrix m
}

I then could write a routine that binds all of my various objects to the structured object:

struct MyStruct MyStructBuild(x,y,a,b,c,m) {
    struct MyStruct scalar S
    
    S.x=x
    S.y=y
    S.a=a
    S.b=b
    S.c=c
    S.m=m
    return(S)
}

Now, I can put all the objects into the function and save the resulting structure:

x=J(1,3,1)
y=runiform(100,100)
a=25
b=.12345
c=1
m="one","two","three"
Z=MyStructBuild(x,y,a,b,c,m)
mata matsave SavedStructure Z

I can then get my structure and everything in it back via:

mata matuse SavedStructure

To get the objects comprising the structure back, I can write a function that returns particular objects, for example:

real matrix MyStruct_x(struct MyStruct S) return(S.x)

One great thing about structures is that is easy to pass a bunch of variables around as one can just pass the structure between functions. Another thing one can do is save matrices of structures, or associative arrays that contain structures. The Mata manual M-2 has a pretty complete description as to how it all works. Anyways, I hope that helps!

Matt Baker

p.s. One thing that is really great about Python that I wish Mata allowed is one can reference parts of the structure interactively. That is, I can type things like S.x and S.y at the command prompt in Python and get back what is wanted. Mata (and I think a lot of other languages are this way as well) doesn't allow this!

[Frank Erickson]

Oh wow, thanks, that's exactly what I was looking for! I had seen matsave before but thought it only applied to matrices. When I type mata describe, I see that my associative array is already classified as a struct, so directly saving it works great, but I'll keep in mind that functions for creating and accessing structures are also an option. As I have two objects, maybe I'll even use it here (to save/load them together).

Aside. Yeah, I'm planning to learn Python when I have time. My most advanced knowledge is of R, which may have its downsides, but has nice syntax like S$x to access items in a "list" (which is analogous to a struct in that it can hold anything). Thanks again!