Forvalues loop to populate matrix in mata

[George D]

Hi all,

I am very new to mata and ran into a problem that I cannot figure out.

I created a matrix m (10x7) in mata and I am trying to populate the individual cells in matrix m from a forvalues loop in stata. I created a mata function called insert() so that I can call it in the forvalues loop but the function does not work as I intend it to. I would really appreciate the help as I am not sure what I am doing wrong. Thank you for the help! Here is the code I used and the error I get:

mata
mata clear
m = J(10,7,0)
void insert(a) {
       m[a,1] = a
}
end

forvalues i=1/10 {
       mata insert(`i')
​}

insert(): 3301 subscript invalid
<istmt>: - function returned error

[James Fiedler]

You'll want to declare that m is an external variable, as in the code below. Look for "external" in help [M-2] declarations.

mata
mata clear
m = J(10,7,0)
void insert(a) {
    external m
    m[a,1] = a
}
end

forvalues i=1/10 {
    mata insert(`i')
}

mata m

[Anthony Thomas]

try passing the matrix m to the function as an argument:

mata:
function insert(m,a){
  m[a, 1] = a
}

m = J(10, 7, .)

end

forval i = 1/10{
   mata: insert(m, `i')
}

I hope that helps!

[Anthony Thomas]

Also just to clarify, that will just fill the first column of "m" with the numbers 1-10 in sequence, which could alternatively be done as m[., 1] = 1::10.

[James Fiedler]

In case it helps, here is what (I think) is happening to produce the error. Mata is permissive about undeclared variables, and in your original code it thinks you want m to be a new variable local to the function. You didn't explicitly create m, so m becomes the default undeclared variable, a transmorphic matrix with zero rows and zero columns.

You can verify this with

mata
mata clear
m = J(10,7,0)
void insert(a) {
    rows(m)
    cols(m)
    m[a,1] = a
}
end

forvalues i=1/10 {
    mata insert(`i')
}

The output from that is

  0
  0
                insert():  3301  subscript invalid
                 <istmt>:     -  function returned error
r(3301);

The zeros are rows(m) and cols(m).

The "subscript invalid" error comes from trying to index row 1 of a matrix with zero rows.

[George D]

Thanks for the help guys! This makes a lot more sense now.

[George D]

You'll want to declare that m is an external variable, as in the code below. Look for "external" in help [M-2] declarations.

mata
mata clear
m = J(10,7,0)
void insert(a) {
external m
m[a,1] = a
}
end

forvalues i=1/10 {
mata insert(`i')
}

mata m

After implementing the solution I ran into another seemingly basic problem that I just could not figure out. I am trying to create a for loop within the void insert() function where I insert a bunch of values into matrix m. Currently my code looks like the following and it works fine but I was wondering if I can create a loop to enter the values for z1-z10. I tried to do the for loop below but it doesn't seem to work. What's the proper syntax to do this? Thank you!

for(i=1;i<=10;i++) {
m[X,i+1] = zi
}

mata
mata clear
m = J(10,11,0)
void insert(X,z1,z2,z3,z4,z5,z6,z7,z8,z9,z10) {
external m
m[X,1] = X
m[X,2] = z1
m[X,3] = z2
m[X,4] = z3
m[X,5] = z4
m[X,6] = z5
m[X,7] = z6
m[X,8] = z7
m[X,9] = z8
m[X,10] = z9
m[X,11] = z10
}
end

[Nick Cox]

You seem to be thinking that Mata will read your syntax zi as if i were like a local macro in Stata. Not only will that not work, I don't think anything in the definition of Mata should lead you to suppose even that it should work. Minimally, i in one context is the name of a numeric scalar, and in another it is part of a scalar name, and there's no looseness about such conversion such as is allowed with macro manipulations in Stata.

The problem here is posed as putting a series of scalars into an array. If that's what you want, I think we need the larger context of how these scalars come into being at all. There are many simpler ways of putting them into a vector. Calling up Mata in a loop to push values one at a time into a vector on the face of it does not seem necessary or appealing. I can't see that there is a need here to write a Mata function at all, but the real problem is why you want to do this.

[George D]

You seem to be thinking that Mata will read your syntax zi as if i were like a local macro in Stata. Not only will that not work, I don't think anything in the definition of Mata should lead you to suppose even that it should work. Minimally, i in one context is the name of a numeric scalar, and in another it is part of a scalar name, and there's no looseness about such conversion such as is allowed with macro manipulations in Stata.

The problem here is posed as putting a series of scalars into an array. If that's what you want, I think we need the larger context of how these scalars come into being at all. There are many simpler ways of putting them into a vector. Calling up Mata in a loop to push values one at a time into a vector on the face of it does not seem necessary or appealing. I can't see that there is a need here to write a Mata function at all, but the real problem is why you want to do this.

I called up Mata to circumvent the matrix size limitations that Stata has. So instead of m = J(10,7,0) I actually have a matrix m that is much bigger than matsize allows. Essentially I am running a forvalues i=1/100000 and populating the matrix m with calculations obtained from each run of the forvalues loop. Hope this provides some context.

[James Fiedler]

One thing you can do is pass a z matrix, and loop over indices

void insert(X, z) {
    external m
    
    m[X,1] = X
    
    for (i = 1; i <= 10; i++) {
        m[X, i+1] = z[i]
    }
}

You can also do this without a loop

void insert(X, z) {
    external m
    
    m[X,1] = X
    m[X, 2..11] = z
}

For these to work properly you have to be sure that z is a vector of length 10. In the second function z must be a rowvector.

You could use these functions like

local X = 1
mata: m = J(10,11,0)
mata: insert(`X', (1, 2, 3, 4, 5, 6, 7, 8, 9, 10))

There might be better approaches though. For the example given here you can accomplish the same without a function:

local X = 1
mata: m = J(10,11,0)
mata: m[`X', 1] = 1
mata: m[`X', 2..11] = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

[Nick Cox]

I called up Mata to circumvent the matrix size limitations that Stata has. So instead of m = J(10,7,0) I actually have a matrix m that is much bigger than matsize allows. Essentially I am running a forvalues i=1/100000 and populating the matrix m with calculations obtained from each run of the forvalues loop. Hope this provides some context.

That helps; but it underlines that a fresh call to Mata 100,000 times over is something that can be avoided. For example, you could store 100,000 values in a Stata variable (use in not if for efficiency) and then if and only if you need to work on that vector within Mata, read the variable into Mata in one go.