Passing "lists of lists of variables" from Stata to Mata

[Yannick Guyonvarch]

Dear Statalisters,

My problem is the following: Assume I want to pass a list of several variable names' lists from Mata to Stata. The following bit of code is an example of what I have in mind:

clear all

set obs 1000

local list_of_lists ""

set seed 1
gen x1=rnormal()
gen x2=rnormal()

scalar a=`""x1""'
local list_of_lists "`list_of_lists' a"

scalar b=`""x1 x2""'
local list_of_lists "`list_of_lists' b"

Then how can I extract each list of variables in Mata to build matrices using st_view or st_data? As a matter of fact, I am not aware of anything as straightforward as what is natural in the case of a unique list of variables' names, where

mata: X=st_data(.,"x1 x2")

is possible.

Regards,

Yannick Guyonvarch

[JanDitzen]

I am not sure if I understand your problem correctly and why you define the scalars a and b. Do you want to use the local list_of_lists as a variablelist?

In the interactive mode you can use locals in mata code. Changing your code a bit to:

clear all

set obs 1000

local list_of_lists ""

set seed 1
gen x1=rnormal()
gen x2=rnormal()

local list_x1x2 "x1 x2"
local list_x1 "x1"

mata: X1X2 = st_data(.,"`list_x1x2'")
mata: X1 = st_data(.,"`list_x1'")
**or
mata:
      X2 = st_data(.,"`list_x1'")
end

This example works in the interactive mode, but not if you work with mata functions. The two locals list_x1x2 and list_x1 contain the the names of the variables and are passed to mata. This works because Stata evaluates the lines first and then calls mata.

Hope this helps.
Jan

[Yannick Guyonvarch]

Thanks Jan for your quick answer. My question was not clear enough I think: I want to work with a varying number of lists of variables so that the only possibility to build the matrices in Mata is to pass the number of variables lists to Mata and to populate matrices with each variables' list in a loop. Since the number of lists is changing, the only way I can think of to pass all the lists to Mata is to build a local with all the variables' lists inside, which is what I tried to explain and highlight in my toy piece of code.

Yannick

[JanDitzen]

I am still not sure. Do you mean something like this:

clear all

set obs 10

set seed 1
gen x1=rnormal()
gen x2=rnormal()

local list1 "x1 x2"
local list2 "x1"

local acc_list "list1 list2"

foreach list in `acc_list' {
    mata `list' = st_data(.,"``list''")
    disp "List: `list' ; with variables: ``list''"
    mata `list'
}

This gives me:

List: list1 ; with variables: x1 x2
                   1              2
     +-------------------------------+
   1 |   .9417381289   -.9382564864  |
   2 |   .4870331128    1.467029664  |
   3 |   .5545321358   -2.796935356  |
   4 |     -.5739419    .6567298696  |
   5 |  -1.683185931   -.0749783493  |
   6 |   .2000260457   -.6136219389  |
   7 |   2.053563013   -1.341230374  |
   8 |  -1.287490564    .4594351343  |
   9 |   .7676956128    1.146460467  |
  10 |   .5712904527    1.376888626  |
     +-------------------------------+
List: list2 ; with variables: x1
                   1
     +----------------+
   1 |   .9417381289  |
   2 |   .4870331128  |
   3 |   .5545321358  |
   4 |     -.5739419  |
   5 |  -1.683185931  |
   6 |   .2000260457  |
   7 |   2.053563013  |
   8 |  -1.287490564  |
   9 |   .7676956128  |
  10 |   .5712904527  |
     +----------------+

[Yannick Guyonvarch]

Yes, that's exactly what I mean! Do you know whether something analogous can be done when passing the "list of lists" as an argument to a Mata function and then looping over lists within the Mata function (this is the ultimate level of refinement I want to reach ^^)?

[Christophe Kolodziejczyk]

Yannick Guyonvarch
Define an array of pointer to store the data from each list. Define an array of strings of the same size containing the local names (Use tokens to this end). Each cell of the array has to point to the result of st_data() with the corresponding list. Use st_local() to read the content of each list
Hope this is helpful

mata:
pointer rowvector passVarlist(string scalar vlist)
{
rowvector string rv_vlist
pointer rowvector data
real scalar i

rv_vlist = tokens(vlist)
data = J(1,cols(rv_vlist),NULL)

for (i =1 ; i <= cols(rv_vlist) ; i++)
{
[INDENT=2]data[i] = &st_data(.,st_local(rv_vlist[i])[/INDENT]

}


return(data)
}

a = passVarlist("list1 list2")
(*a[1])[1,1] // gives the first element of your matrix of variables from list1

end

[Yannick Guyonvarch]

Hello Christophe,

Thanks for the answer, that's perfect! I had not thought of the st_local() command, I think that's what I was missing.

[Yannick Guyonvarch]

Unfortunately I was not able to make your code section work... However the following (slight) modification of your code works for me.

clear all

set obs 1000
set more off

gen x1=rnormal()
gen x2=rnormal()

local list_of_lists ""

scalar a=`""x1""'
local list_of_lists "`list_of_lists' a"

scalar b=`""x1 x2""'
local list_of_lists "`list_of_lists' b"


mata:
pointer rowvector passVarlist(string scalar vlist) {

    string rowvector rv_vlist
    pointer(real matrix) rowvector data
    real scalar i
    
    rv_vlist = tokens(st_local(vlist))
    
    data = J(1,cols(rv_vlist),NULL)
    
    for (i =1 ; i <= cols(rv_vlist) ; i++) {
    
        data[i] = &st_data(.,tokens(st_strscalar(rv_vlist[i])))
    }
    
    return(data)
}

a = passVarlist("list_of_lists")
(*a[1]) // prints the first matrix of variables from list_of_lists
(*a[2]) // prints the second matrix of variables from list_of_lists
end

[Christophe Kolodziejczyk]

Yannick Guyonvarch Ok. What was the error? Actually I hadn't tested the program. A parenthesis was missing but I guess it was there the problem was
Normally is not required to pass a vector of strings to st_data(), a list variables in a string scalar separated with blanks should work fine

[Yannick Guyonvarch]

Hello Christophe. There was another small bug in the code which was that it read "rowvector string" instead of "string rowvector" but once I changed that, your code worked. Sorry about that. Still, I can't pass "list1 list2" as an argument to my mata program since I don't know in advance how many lists I will have to deal with (I use the mata program within an ado program whose behaviour depends on what user supplies). Therefore I can only pass as an argument a unique local that is incremented with the desired number of lists of variables and in that case typing a = passVarlist("list_of_lists") with list_of_lists a local that contains"list1 list2" for instance (where list1 and list2 are locals that store lists of variable names) does not work anymore. My approach works with a changing number of lists but it may not be an optimal way of coding it...