passing a class instance between Stata and Mata

[Salah Mahmud]

Hello,

I have a need to pass an instance of a class created in Mata to Stata and then get it back from Stata. I tried different approaches but each was blocked by one Mata limitation or another:

1. The most straightforward solution is to return a transmorphic scalar/matrix (or a pointer to it) to Stata. But st_numscalar, st_local etc only accept a real or string parameter, and there is no way that I could find to convert between transmorphic scalar/matrix (or a pointer to it) and real or string scalar.

2. Another solution is to hold the instances in an asarray/vector in Mata and pass to Stata a handle (real scalar) that when passed back by Stata can be used to retrieve the corresponding instance from the asarray/vector. The obstacle here is that the asarray/vector must persist between function calls, and I could not find a way to create a global Mata variable (AFAIK, Mata does not support declaring variables outside functions) or the equivalent of C static functions.

3. Also tried to create a class with a public static transmorphic scalar as a member and a static accessor function, but any assignment to the static member in the static function results in error 3000 invalid lval.

Does anyone know a way around or possibly a totally different solution.

Thanks,

[Mike Lacy]

Yes, you can have variables defined outside a function and they persist in Mata even if you leave it and come back. Here's a simple example which I hope is relevant to your needs:
// In Stata

mata mata clear
mata:
X = 999
function myfcn (
    real scalar whatever)
{
   external real scalar X
   printf("Hello world.  My global X = %5.0f\n", X)
}
myfcn(X)
end
display "Now we're back in Stata."
mata: printf("Persisting across calls to Mata, my global X still = %5.0f\n", X)

See the entry on "external variable" in [M-6] Glossary.

[Salah Mahmud]

Hi Mike,
Of course you are right.
What threw me off is that I kept getting an error if I tried to declare a variable outside a Mata function (something vague like “something found when nothing was expected!!”).
<rant> It is unexpected behaviour to be able to initialize a variable before you declare it (when matastrict is on), but that is exactly the syntax required for using ‘external’ variables. Not to mention that the label ‘external’ is misleading (at least in the C sense of extern which means that the variable/function is defined elsewhere e.g., in an external library). A variable defined in the same file is hardly external </rant>.

Thanks for the insights, Mike,

/salah

[Joseph Coveney]

You don't need a static member function in order to accomplish what you want. You can do it with just a static member variable. See below.

The illustration has a probe class ("KeepMe") that is instantiated and then set with a tell-tale message in order to distinguish it as the instantiated object later. After it is instantiated and ID'd, it is stored in an instantiated object of a storage class ("KeepEm") that keeps the probe object (or rather a pointer to it) in a static member variable that is set and accessed via conventional (not static) member functions. This avoids the problem that you were experiencing with static member functions, and yet is able to retain an instantiated object through a journey from Mata to Stata and back.

This approach will accomplish your objective without resorting to global variables and their problems.

In the illustration below, after the sojourn to Stata and back, the original instantiated object is retrieved from another instantiated object of the storage class (the probe object is still there, because the member variable is static) via a conventional (not static) accessor function, and then it's tested to prove that it's the original object that was instantiated before the trip to Stata and back.

version 15.0

clear *

mata:
mata set matastrict on

class KeepMe {
    private:
        string scalar message
    public:
        final void setMessage()
        final void reportMessage()
}
void function KeepMe::setMessage(string scalar message) this.message = message
void function KeepMe::reportMessage() printf(message + "\n")

class KeepEm {
    private:
        static pointer(class KeepMe scalar) scalar keepme_pointer
    public:
        final void setKeepMe()
        final class KeepMe scalar getKeepMe()
}
void function KeepEm::setKeepMe(class KeepMe scalar keepme) keepme_pointer = &keepme
class KeepMe scalar function KeepEm::getKeepMe() return(*keepme_pointer)

void function createm() {
    class KeepMe scalar keepme
    keepme.setMessage("Hello, world!")

    class KeepEm scalar keepem
    keepem.setKeepMe(keepme)
}

void function testem() {
    class KeepMe scalar keepme
    keepme.setMessage("You should see 'Hello, world!' alone if you get back " +
        "the original instantiated object and not this message.")

    class KeepEm scalar keepem
    keepme = keepem.getKeepMe()

    // Here's the proof:
    keepme.reportMessage()
}

// Set up and exit to Stata
createm()

end

// Now in Stata; now back to Mata for the test

mata:
mata set matastrict on

testem()

end

exit

Note that it's important that the data type of the storage member variable be a pointer to the class and not the class, itself.

version 15.0

clear *

mata:
mata set matastrict on

class A {
}

class BPointer {
    private:
        pointer(class A scalar) scalar a
    public:
        final void setA()
}
void function BPointer::setA(class A scalar a) this.a = &a

class BClassItself {
    private:
        class A scalar a
    public:
        final void setA()
}
void function BClassItself::setA(class A scalar a) this.a = a

a = A()
b = BPointer()
b.setA(a)

b = BClassItself()
b.setA(a)

end

exit

[Attaullah Shah]

Joseph Coveney
I have greatly benefited from your replies to questions related to class programming. In the above code, you have shown great dexterity. For my understanding, can you comment on how the following approach is different from what you have suggested. I am interested in knowing how your approach delivers more utility. This is what I normally do.
1. Instantiate an object inside Stata program
2. Set its elements through getter functions inside Stata
3. Pass the instantiated object from Stata to Mata using a function
4. Use the class attributes / members inside Mata

[Alfonso Sánchez-Peñalver]

Looking at Joseph Coveney's first set of code I find something very troubling. The function createm() is called first, and creates a new instance of the KeepEm and names it keepem, to then set the pointer to the object that holds the message. In the function testem() he, once again, creates a new instance of the KeepEm and, again, names it keepem. At this point the old instance should have been wiped out of memory, and the new instance should have an empty pointer in keepme_pointer, but it doesn't. I find this extremely troubling, because I strongly believe that the second declaration of the object with the same name should completely wipe out the first instance of the object.

[Joseph Coveney]

At this point the old instance should have been wiped out of memory, and the new instance should have an empty pointer in keepme_pointer, but it doesn't. I find this extremely troubling, because I strongly believe that the second declaration of the object with the same name should completely wipe out the first instance of the object.

I think that there's a couple of things going on here.

First, yes, the name keepem is the same for the two objects, but the two usages of the same name or token are not within the same scope. They're in separate functions, and so despite incidentally sharing the same name or token, the objects are separate and won't wipe each other out.

Second, the pointer member variable is declared static and so all contemporary instances of the class KeepEm will share the same pointer variable pointing to the same instance of KeepMe.

[Joseph Coveney]

This is what I normally do.
1. Instantiate an object inside Stata program
2. Set its elements through getter functions inside Stata . . .

I'm not sure how you do that. There is a second object-oriented programming facility that's actually the first, i.e., long before Mata's.

help classman

But I think that its objects are not interchangeable with those created in Mata.

So, I'm a bit confused as to what you're saying. Are you saying that you create a Mata object in Mata code inside an ado-file?

[Attaullah Shah]

Joseph Coveney
This is how I do it.

prog testclass
syntax [anything], [font(str)]

    mata : mft = FormatClassAsdocx()
    mata : mft.font_family = "`font'"

    mata : displaydet(mft)

end

mata
class FormatClassAsdocx {

        string scalar font_family
}
void displaydet(class FormatClassAsdocx scalar mft){
    "You have use the " + mft.font_family + " Font"
}
end

To test it

 testclass, font(Verdana)
  You have use the Verdana Font

To rephrase, my question is how the use of pointers can deliver more utility in this context. Second, if someone / program clears the mata memory, will the pointers still exist and retrieve the stored information. If not, then how can one protect against clearing mata memory?

If there is no way one can guard against this, are we left with the only option of using globals or there can be other efficient ways to store session information.

[Joseph Coveney]

To rephrase, my question is how the use of pointers can deliver more utility in this context.

I don't know about your question. I used pointers in order to address the OP's question, which is different from yours. The OP was interested in going from Mata to Stata and then back again. Yours seems to be a standard ado-file with a standard Mata component. I think that such a conventional Stata-Mata mixed application doesn't often require pointers.

Second, if someone / program clears the mata memory, will the pointers still exist and retrieve the stored information.

I assume that clearing Mata's memory will clear Mata's memory, including pointers, although I haven't specifically looked into it.

If not, then how can one protect against clearing mata memory?

If there is no way one can guard against this, are we left with the only option of using globals

Again, I assume that deliberately clearing Mata's memory is not intended or designed to be protected against, and that clearing Mata's memory will clear Mata's memory, including Mata's global variables. So, I'm not sure that global variables are an option here.

[Joseph Coveney]

I think that there's a couple of things going on here.

I've illustrated both phenomena below: same object name in different scope of execution & sharing pointed-to object in static member variable between two instances of storage object.

.ÿ
.ÿversionÿ16.1

.ÿ
.ÿclearÿ*

.ÿ
.ÿlocalÿline_sizeÿ`c(linesize)'

.ÿsetÿlinesizeÿ80

.ÿ
.ÿmata:
-------------------------------------------------ÿmataÿ(typeÿendÿtoÿexit)ÿ------
:ÿmataÿsetÿmatastrictÿon

:ÿ
:ÿclassÿProbeÿ{
>ÿÿÿÿÿprivate:
>ÿÿÿÿÿÿÿÿÿstringÿscalarÿname
>ÿÿÿÿÿÿÿÿÿvoidÿnew()
>ÿÿÿÿÿpublic:
>ÿÿÿÿÿÿÿÿÿfinalÿvoidÿset()
>ÿÿÿÿÿÿÿÿÿfinalÿstringÿscalarÿget()
>ÿ}

:ÿvoidÿfunctionÿProbe::new()ÿnameÿ=ÿ"Default"

:ÿvoidÿfunctionÿProbe::set(stringÿscalarÿname)ÿthis.nameÿ=ÿname

:ÿstringÿscalarÿProbe::get()ÿreturn(name)

:ÿ
:ÿclassÿStorage{
>ÿÿÿÿÿprivate:
>ÿÿÿÿÿÿÿÿÿstaticÿpointer(classÿProbeÿscalar)ÿscalarÿprobe_ptr
>ÿÿÿÿÿpublic:
>ÿÿÿÿÿÿÿÿÿfinalÿvoidÿstore()
>ÿÿÿÿÿÿÿÿÿfinalÿclassÿProbeÿscalarÿretrieve()
>ÿ}

:ÿvoidÿfunctionÿStorage::store(classÿProbeÿscalarÿprobe)ÿprobe_ptrÿ=ÿ&probe

:ÿclassÿProbeÿscalarÿfunctionÿStorage::retrieve()ÿreturn(*probe_ptr)

:ÿ
:ÿclassÿTestÿ{
>ÿÿÿÿÿprivate:
>ÿÿÿÿÿÿÿÿÿfinalÿvoidÿelsewhere()
>ÿÿÿÿÿpublic:
>ÿÿÿÿÿÿÿÿÿfinalÿvoidÿsameScope()
>ÿÿÿÿÿÿÿÿÿfinalÿvoidÿdifferentScope()
>ÿ}

:ÿvoidÿfunctionÿTest::elsewhere()ÿ{
>ÿÿÿÿÿclassÿProbeÿscalarÿoriginalÿ//ÿ<-ÿsameÿnameÿforÿobject:ÿ"original"
>ÿÿÿÿÿoriginal.set("Outÿofÿscope")ÿ//ÿ<-ÿbutÿyouÿwon'tÿseeÿthisÿmessage
>ÿ}

:ÿvoidÿfunctionÿTest::sameScope()ÿ{
>ÿ
>ÿÿÿÿÿclassÿProbeÿscalarÿoriginal
>ÿÿÿÿÿoriginal.set("NotÿDefault")
>ÿ
>ÿÿÿÿÿclassÿStorageÿscalarÿs
>ÿÿÿÿÿs.store(original)
>ÿ
>ÿÿÿÿÿprintf("Verifyingÿoriginal:ÿ\nÿ%-10s\n\n",ÿs.retrieve().get())
>ÿ
>ÿÿÿÿÿ//ÿOverwritingÿbyÿusingÿsameÿnameÿwithinÿsameÿscope;ÿyou'llÿseeÿ"Default"
>ÿÿÿÿÿoriginalÿ=ÿProbe()
>ÿ
>ÿÿÿÿÿprintf("Sameÿscope->overwritten:\nÿ%-10s\n\n",ÿs.retrieve().get())
>ÿ}

:ÿvoidÿfunctionÿTest::differentScope()ÿ{
>ÿ
>ÿÿÿÿÿclassÿProbeÿscalarÿoriginal
>ÿÿÿÿÿoriginal.set("NotÿDefault")
>ÿ
>ÿÿÿÿÿclassÿStorageÿscalarÿs
>ÿÿÿÿÿs.store(original)
>ÿ
>ÿÿÿÿÿ//ÿHere,ÿsameÿname,ÿbutÿdifferentÿscope;ÿtheÿmessageÿstaysÿ"NotÿDefault"
>ÿÿÿÿÿelsewhere()
>ÿ
>ÿÿÿÿÿprintf("Differentÿscope->notÿoverwritten:\nÿ%-10s\n\n",ÿs.retrieve().get()
>ÿ)
>ÿ}

:ÿ
:ÿtestÿ=ÿTest()

:ÿtest.sameScope()
Verifyingÿoriginal:ÿ
ÿNotÿDefault

Sameÿscope->overwritten:
ÿDefaultÿÿÿ


:ÿ
:ÿtest.differentScope()
Differentÿscope->notÿoverwritten:
ÿNotÿDefault


:ÿ
:ÿnew_storageÿ=ÿStorage()

:ÿprintf("%-10s\n\n",ÿnew_storage.retrieve().get())ÿ//ÿ<-ÿstaticÿmemberÿfunction
NotÿDefault


:ÿ
:ÿend
--------------------------------------------------------------------------------

.ÿ
.ÿsetÿlinesizeÿ`line_size'

.ÿ
.ÿexit

endÿofÿdo-file


.

I've attached the do-file, itself, if you want to explore things on your own, for example, the fact that the pointer is still holding an object I think prevents the first instance from being destroyed despite the function's going out of scope that created it. You'd probably want to put a destructor in the class if you really wanted to get rid of it (or, as mentioned above, forcibly clearing Mata's memory).