Variance-covariance matrix not symmetric

[Alfonso Sánchez-Peñalver]

Hi everyone,

I'm having an issue when forming the variance-covariance matrix from a vector with the standard deviations and a correlation matrix. The attached picture shows the work and results. S holds the standard deviations for seven random parameters. R is the correlation matrix which is definitely symmetric but, although the variance-covariance matrix looks symmetric it is not. This looks as an issue of precision, and I was wondering if there was a way in mata to get it right.

Thanks,

Alfonso.

[Sebastian Kripfganz]

It is indeed an issue of precision. To circumvent such a problem, you can enforce symmetry as follows:

V = diag(S) * R * diag(S)
_makesymmetric(V)

or

V = makesymmetric(diag(S) * R * diag(S))

[Alfonso Sánchez-Peñalver]

Thank you Sebastian. But the loss of precision in the product is exactly what worries me.

[John Mullahy]

Alfonso: I'm wondering if you might have tried something like this

dS=diag(S)
V=quadcross(dS,(R*dS))
issymmetric(V)

[John Mullahy]

P.S. I tried this a few different ways and it appears that quadcross doesn't solve the problem. Sorry.

[Alfonso Sánchez-Peñalver]

No worries John. Thanks for the suggestion. It’s troubling because this is to use with maximum likelihood for the estimation of the parameters in a multivariate normal distribution and I would have liked to get to the standard deviations and correlations instead of the elements of the cholesky decomposed variance covariance matrix, which is how I have it currently. Thanks again!!!

[Joseph Coveney]

If it's any consolation, it looks like the elementwise differences are down around 10^-20 or so.

Would a split-the-difference approach help any? It yields a symmetric matrix at the midpoint of the element pairs that have any difference. It might be a little more tenable than makesymmetric(), which just substitutes the upper triangle with elements from the lower.

:ÿSÿ=ÿ.008262881,ÿ.0420706912,ÿ.0005536591,ÿ.113492093,ÿ.2922291694,ÿ.0057515581,ÿ.430931134

:ÿRÿ=ÿJ(7,ÿ7,ÿ0.1)ÿ+ÿI(7)ÿ*ÿ0.9

:ÿVÿ=ÿdiag(S)ÿ*ÿRÿ*ÿdiag(S)

:ÿV
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿÿÿÿÿÿ2ÿÿÿÿÿÿÿÿÿÿÿÿÿ3ÿÿÿÿÿÿÿÿÿÿÿÿÿ4ÿÿÿÿÿÿÿÿÿÿÿÿÿ5ÿÿÿÿÿÿÿÿÿÿÿÿÿ6ÿÿÿÿÿÿÿÿÿÿÿÿÿ7
ÿÿÿÿ+---------------------------------------------------------------------------------------------------+
ÿÿ1ÿ|ÿÿ.0000682752ÿÿÿ.0000347625ÿÿÿ4.57482e-07ÿÿÿ.0000937772ÿÿÿ.0002414655ÿÿÿ4.75244e-06ÿÿÿ.0003560733ÿÿ|
ÿÿ2ÿ|ÿÿ.0000347625ÿÿÿ.0017699431ÿÿÿ2.32928e-06ÿÿÿ.0004774691ÿÿÿ.0012294283ÿÿÿ.0000241972ÿÿÿ.0018129571ÿÿ|
ÿÿ3ÿ|ÿÿ4.57482e-07ÿÿÿ2.32928e-06ÿÿÿ3.06538e-07ÿÿÿ6.28359e-06ÿÿÿ.0000161795ÿÿÿ3.18440e-07ÿÿÿ.0000238589ÿÿ|
ÿÿ4ÿ|ÿÿ.0000937772ÿÿÿ.0004774691ÿÿÿ6.28359e-06ÿÿÿ.0128804552ÿÿÿÿÿ.00331657ÿÿÿ.0000652756ÿÿÿ.0048907276ÿÿ|
ÿÿ5ÿ|ÿÿ.0002414655ÿÿÿ.0012294283ÿÿÿ.0000161795ÿÿÿÿÿ.00331657ÿÿÿ.0853978874ÿÿÿ.0001680773ÿÿÿ.0125930647ÿÿ|
ÿÿ6ÿ|ÿÿ4.75244e-06ÿÿÿ.0000241972ÿÿÿ3.18440e-07ÿÿÿ.0000652756ÿÿÿ.0001680773ÿÿÿ.0000330804ÿÿÿ.0002478525ÿÿ|
ÿÿ7ÿ|ÿÿ.0003560733ÿÿÿ.0018129571ÿÿÿ.0000238589ÿÿÿ.0048907276ÿÿÿ.0125930647ÿÿÿ.0002478525ÿÿÿ.1857016423ÿÿ|
ÿÿÿÿ+---------------------------------------------------------------------------------------------------+

:ÿDÿ=ÿVÿ:-ÿV'

:ÿD
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ2ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ3ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ4ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ5ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ6ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ7
ÿÿÿÿ+----------------------------------------------------------------------------------------------------------+
ÿÿ1ÿ|ÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿ6.77626e-21ÿÿÿ-5.29396e-23ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿ2.71051e-20ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿ|
ÿÿ2ÿ|ÿÿ-6.77626e-21ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿ-3.38813e-21ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿ|
ÿÿ3ÿ|ÿÿÿ5.29396e-23ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿ3.38813e-21ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿ3.38813e-21ÿÿ|
ÿÿ4ÿ|ÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿ8.67362e-19ÿÿ|
ÿÿ5ÿ|ÿÿ-2.71051e-20ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿ-3.38813e-21ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿ-2.71051e-20ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿ|
ÿÿ6ÿ|ÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿ3.38813e-21ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿ2.71051e-20ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿ|
ÿÿ7ÿ|ÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿ-3.38813e-21ÿÿÿ-8.67362e-19ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ0ÿÿ|
ÿÿÿÿ+----------------------------------------------------------------------------------------------------------+

:ÿV2ÿ=ÿVÿ:-ÿ(Dÿ:/ÿ2)

:ÿV2
[symmetric]
ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ1ÿÿÿÿÿÿÿÿÿÿÿÿÿ2ÿÿÿÿÿÿÿÿÿÿÿÿÿ3ÿÿÿÿÿÿÿÿÿÿÿÿÿ4ÿÿÿÿÿÿÿÿÿÿÿÿÿ5ÿÿÿÿÿÿÿÿÿÿÿÿÿ6ÿÿÿÿÿÿÿÿÿÿÿÿÿ7
ÿÿÿÿ+---------------------------------------------------------------------------------------------------+
ÿÿ1ÿ|ÿÿ.0000682752ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿ2ÿ|ÿÿ.0000347625ÿÿÿ.0017699431ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿ3ÿ|ÿÿ4.57482e-07ÿÿÿ2.32928e-06ÿÿÿ3.06538e-07ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿ4ÿ|ÿÿ.0000937772ÿÿÿ.0004774691ÿÿÿ6.28359e-06ÿÿÿ.0128804552ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿ5ÿ|ÿÿ.0002414655ÿÿÿ.0012294283ÿÿÿ.0000161795ÿÿÿÿÿ.00331657ÿÿÿ.0853978874ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿ6ÿ|ÿÿ4.75244e-06ÿÿÿ.0000241972ÿÿÿ3.18440e-07ÿÿÿ.0000652756ÿÿÿ.0001680773ÿÿÿ.0000330804ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ|
ÿÿ7ÿ|ÿÿ.0003560733ÿÿÿ.0018129571ÿÿÿ.0000238589ÿÿÿ.0048907276ÿÿÿ.0125930647ÿÿÿ.0002478525ÿÿÿ.1857016423ÿÿ|
ÿÿÿÿ+---------------------------------------------------------------------------------------------------+

:ÿissymmetric(V2)
ÿÿ1

[Sebastian Kripfganz]

To be honest, I do not quite understand the discussion. If the precision error is around 10^-20, why bother? There is nothing that makes the error above the main diagonal sytematically smaller than that below the main diagonal (or vice versa), nor does a split-the-difference approach systematically improve the precision. Such a precision error is simply something we have to live with when doing this kind of computation, and the makesymmetric() function is specifically designed to deal with this situation.

Edit: The split-the-difference approach might actually even worsen the situation because taking differences is particularly prone to precision error. You obtain a truly symmetric matrix at the cost of a lower overall precision.

[Alfonso Sánchez-Peñalver]

Sebastian Kripfganz, as I mentioned in #6, the issue comes in a maximum likelihood estimation. The estimation is of a mixed logit via simulated maximum likelihood. I would like to parameterize the natural logarithm of the standard deviations and the inverse hyperbolic tangents of the correlations, to then form the variance-covariance matrix of the mixing distribution (multivariate normal), and then use its cholesky decomposition with halton draws to get the random terms of the parameters. When I do so, ml gives errors with the numeric derivatives. When, instead, I parametrize the elements of the cholesky decomposed variance-covariance matrix, the program works fine. I can't say 100% certain that the issue is with the loss of precision, but it is at least very suspicious that the code works one way and not the other.

[Sebastian Kripfganz]

I would be surprised if this imprecision causes the maximum likelihood estimation to fail. Unless you are explicitly getting an error message saying that a matrix is not symmetric although it should be, I would guess that the problem lies somewhere else.

[John Mullahy]

Alfonso: I'm not sure I'm interpreting correctly your comments in #6 and #9 about the Cholesky decomposition, in particular at what point in your algorithm the decomposition is used. But if you run this Mata code it may offer an illustrative suggestion regarding where the "asymmetry" arises.

mata

s=uniform(7,1)
ds=diag(s)
r=J(7,7,.1)+diag(J(7,1,.9))
cr=cholesky(r)

issymmetric(r)
issymmetric(cr*cr')
issymmetric(ds*r*ds)
issymmetric(ds*cr*cr'*ds)
issymmetric(ds*(cr*cr')*ds)
issymmetric((ds*cr)*(cr'*ds))

end