[Peter Hoon]

Clyde and statalist members: I'm getting invalid syntax r98 error when reading in a lower triangle .txt file of variances - covariances using Clyde's new code for 87 variables. I'm unable to spot where the error is and how it might be corrected.



cd c:\chest
c:\chest

. set more off

. clear

. import delimited c:\Mplus\oct292024montecarlovarcovmtx.txt
(1 var, 766 obs)

.
. // Clyde's revised Dec 6 2024 code follows
. split v1, gen(var) destring
variables born as string:
var1 var2 var3 var4 var5
var1 has all characters numeric; replaced as double
var2 has all characters numeric; replaced as double
var3 has all characters numeric; replaced as double
var4 has all characters numeric; replaced as double
(1 missing value generated)
var5 has all characters numeric; replaced as double
(1 missing value generated)

. drop v1

.
. // CREATE THE MATRIX
. local nvars 87 //Set nvars to the num of variables you have*/

. local r 1

. local c 1

.
. matrix M = I(`nvars')

.
. forvalues o = 1/`=_N' {
2. forvalues v = 1/`ncols' {
3. matrix M[`r', `c'] = var`v'[`o']
4. if `r' != `c' {
5. matrix M[`c', `r'] = M[`r', `c']
6. }
7. local ++c
8. if `c' > `r' {
9. local c = 1
10. if `r' < `nvars' {
11. local ++r
12. }
13. else {
14. continue, break
15. }
16. }
17. }
18. }
invalid syntax
r(198);

end of do-file

r(198);

Here is the complete do code I'm trying to run:


cd c:\chest
set more off
clear
import delimited c:\Mplus\oct292024montecarlovarcovmtx.txt

// Clyde's revised Dec 6 2024 code follows
split v1, gen(var) destring
drop v1

// CREATE THE MATRIX
local nvars 87 //Set nvars to the num of variables you have*/
local r 1
local c 1

matrix M = I(`nvars')

forvalues o = 1/`=_N' {
forvalues v = 1/`ncols' {
matrix M[`r', `c'] = var`v'[`o']
if `r' != `c' {
matrix M[`c', `r'] = M[`r', `c']
}
local ++c
if `c' > `r' {
local c = 1
if `r' < `nvars' {
local ++r
}
else {
continue, break
}
}
}
}
local names // BUILD A LIST OF NAMES TO APPLY TO BOTH ROWS AND COLUMNS
forvalues i = 1/87 { // 87 OR WHATEVER THE DIMENSION OF THE MATRIX IS
local names `names' var`i'
}

// RENAME THE ROWS AND COLULMNS Clyde Schechter's code
matrix rownames M = `names'
matrix colnames M = `names'


local names SINGLE MARRIED SEPARATE DIVORCED WIDOWED
HISORG
IRIBE YR_BIRTH SEQ_NUM MDXRCMP ORGRISK FORORG
CENTRAL UPPERIN LOWERIN UPPEROUT LOWEROUT AXILTAIL
OVERLAP LATERAL GRADE DX_CONF EOD10_SZ EOD10_EX
EOD10_NE ASPSURG CSTUMSIZ CSEXTEN LYMPOD10 CS5SITE
CS6SITE DAJCCT DAJCCN DAJCCSTG SURGPRIF SURGSITF
NUMNODES NO_SURG RADATION RADMIS RAD_SURG SS_SURG
SURGSITE ICD9V10V UNSPPSM EPITHPSM SQUAMPSM ADENCAR
CYTMUSER DUCTLOB HST_STGA A3SEERSG FIRSTPRM CTYMEDN
CTYPOV CTYPOV18 POVBIRTH CTYINCID HSEDSTAT COLEDUST
DXCTYCOL DXCTYHS DXSTRISK CODPUB STAT_REC SUMM2K
DETHCLSS CSTSEVAL CSRGEVAL CSMETVAL INTPRIM ERSTATUS
PRSTATUS SRVTIMON INSRECPB ADJTM6VL CSMETDX CS7SITE
HER2 BRST_SUB METBONPB METBRPB METLVPUB METLGPUB
T_VALUE N_VALUE M_VALUE
// 87 renamed

// give matrix M new names for rows and columns
matrix rownames M = `names'
matrix colnames M = `names'

matrix list M/*list your matrix to confirm accuracy of import and read in.
As Clyde points out, the number of variables in your data set
is equal to the dimension of the matrix*/

matrix symeigen X v = M
matrix list v/*list eigenvalues to spot a non positive definite
variance - covariance mtx.*/

[Clyde Schechter]

I see the problem. In #15 I posted code that began with generating a demonstration data set, followed by code to actually create the matrix. And, unfortunately, I explicitly said that you only needed the part that followed // CREATE THE MATRIX. That, however, turns out not to be true. It's true except for one thing. The local macro ncols was defined up in the part of the code that created the demonstration data set. But local macro ncols is still needed in the matrix creation code. By running the matrix creation code separately, the definition of local macro ncols never happend. Consequently the line -forvalues v = 1/`ncols' {- becomes a syntax error because ncols is undefined.

The solution is simple: Insert a new line of code -local ncols 5- immediately after the line -matrix M = I(`nvars')-. That will fix this problem.

My apologies for the confusion.

[Peter Hoon]

Clyde, deeply appreciate your support in revising for 87 variables.. A big boost to my work. Here is the revised code for computing eigenvalues which statalist members may find helpful:

/*
Clyde Schechter's code for reading in a var - cov mtx written out
by Mplus as a .txt file.  Mplus's .txt file is all one 
single dimensioned vector,
white space delimited, representing a
a lower triangle variance - covariance matrix. Most likely,
SAS, SPSS R, AMOS also write out variance covariance matrices in .txt.
The code below
gets the .txt file that Mplus outputs into a matrix that can be read into
Stata factor analysis program factormat, 9/13/2023.
Stata's factormat program starts with "summary data",
a variance - covariance matrix, which several authors
(Kline, Bollen and Bentler) recommend
(in lieu of a correlation matrix) for SEM and factor analysis work.
The Matrix M looks like this as it is read from left to right,
as Clyde points out, "snake like", into Stata:

var
cov var
cov cov var
cov cov cov var etc.

Stata factormat with quartimax rotation runs in just a few seconds,
a great advantage in gaining a rapid understanding of latent constructs.
Stata _getcovcorr will alter the covariance matrix back into a correlation matrix,
that Mplus (and others) can read and use to build ESEM or CSEM models. Eventhough,
a covariance matrix is recommended, Mplus often requires
a correlation matrix for ESEM, EFA, etc.

An important point to remember, that is often confusing
in Stata:
The lower triangle matrix is read into Stata correctly.
Yet the matrix is stored as a full Hermitian symmetric matrix by
Stata in memory. The Stata command 
matrix list M
DISPLAYS a lower triangle
matrix! You are led to believe it is stored as such, but it is not.
To rtepeat, it is stored as full symmetric Hermitian.

A covariance, SEM or FA model on a hex core machine
running at 3.6 GHz,
turbo boosted,
can take up to 60 hours in Mplus, SAS, SPSS or Stata from raw data,
when observations are large (seven hundred thousand). 

When summary
data is used as input, EFA, SEM, CFA, CSEM  in Stata
may take only A FEW SECONDS from a matrix of
dimensioned at 87. The importance/advantage of having the capability
of reading in a var - cov mtx from SAS, Mplus, Stata, SPSS, AMOS shown
in the output below, becomes 
clear for multivariate researchers who have big data.
*/

cd c:\chest
set more off
clear
import delimited c:\Mplus\oct292024montecarlovarcovmtx.txt

// Clyde's revised Dec 6 2024 code follows
split v1, gen(var) destring
drop v1

//  CREATE THE MATRIX
local ncols 5 
local nvars 87 //Set nvars to the num of variables you have
local r 1
local c 1

matrix M = I(`nvars')  

forvalues o = 1/`=_N' {
    forvalues v = 1/`ncols' {
        matrix M[`r', `c'] = var`v'[`o']
        if `r' != `c' {
            matrix M[`c', `r'] = M[`r', `c']
        }
        local ++c
        if `c' > `r' {
            local c = 1
            if `r' < `nvars' {
                local ++r
            }
            else {
                continue, break
            }
        }
    }
}
local names // BUILD A LIST OF NAMES TO APPLY TO BOTH ROWS AND COLUMNS
forvalues i = 1/87 { // 87 OR WHATEVER THE DIMENSION OF THE MATRIX IS
    local names `names' var`i'
}

// RENAME THE ROWS AND COLULMNS Clyde Schechter's code
matrix rownames M = `names'
matrix colnames M = `names'
            
            
local names SINGLE      MARRIED     SEPARATE    DIVORCED    WIDOWED HISORG ///
     IRIBE       YR_BIRTH    SEQ_NUM     MDXRCMP     ORGRISK     FORORG ///      
     CENTRAL     UPPERIN     LOWERIN     UPPEROUT    LOWEROUT    AXILTAIL ///    
     OVERLAP     LATERAL     GRADE       DX_CONF     EOD10_SZ    EOD10_EX ///   
     EOD10_NE    ASPSURG     CSTUMSIZ    CSEXTEN     LYMPOD10    CS5SITE ///     
     CS6SITE     DAJCCT      DAJCCN      DAJCCSTG    SURGPRIF    SURGSITF ///    
     NUMNODES    NO_SURG     RADATION    RADMIS      RAD_SURG    SS_SURG ///     
     SURGSITE    ICD9V10V    UNSPPSM     EPITHPSM    SQUAMPSM    ADENCAR ///     
     CYTMUSER    DUCTLOB     HST_STGA    A3SEERSG    FIRSTPRM    CTYMEDN ///     
     CTYPOV      CTYPOV18    POVBIRTH    CTYINCID    HSEDSTAT    COLEDUST ///    
     DXCTYCOL    DXCTYHS     DXSTRISK    CODPUB      STAT_REC    SUMM2K ///      
     DETHCLSS    CSTSEVAL    CSRGEVAL    CSMETVAL    INTPRIM     ERSTATUS ///    
     PRSTATUS    SRVTIMON    INSRECPB    ADJTM6VL    CSMETDX     CS7SITE ///     
     HER2        BRST_SUB    METBONPB    METBRPB     METLVPUB    METLGPUB ///    
     T_VALUE     N_VALUE     M_VALUE 
// 87 renamed
        
// give matrix M new names for rows and columns
matrix rownames M = `names'
matrix colnames M = `names'

matrix list M //list your matrix to confirm accuracy of import and read in.
              //As Clyde points out, the number of variables in your data set
              //is equal to the dimension of the matrix

matrix symeigen X v = M
matrix list v //list eigenvalues to spot a non positive-definite
              //variance - covariance mtx. In current case last
              //eigenvalue is very slightly negative (10 to minus 6)