Difference between xttrans and xttrans2 for constructing transition matrices

[Jessica Thacker]

(i) I would like to ask the exact difference between xttrans and xttrans2.
I had read the help file for xttrans and xttrans2 where it said "xttrans2, another generalization of tabulate, reports transition probabilities. It differs from
official Stata's xttrans only in detail: (a) options of tabulate other than row and freq also may be specified; (b) a matrix of probabilities may be saved; (c) it may not be called by varlist". (1a)What is the exact difference between the two? When I run xttrans and xttrans2, I get the same results. (1b) And, even xttrans, freq and xttrans2, freq works out to give the same result which means that part (a) of (i) is not the difference between xttrans and xttrans2?

(ii) I have unbalanced panel data of firms from year 2011-2015, which I balanced using the fillin command . using the help manual of xttrans I found that "Thus if the data are fully rectangularized, xttrans produces (inefficient) estimates of the Markov transition matrix. " What does this mean? And, what are rectangularized datasets?

(iii) I have a panel of firms from 2011-2020, with panel id "cocode" and year "Year". I wish to find how is the regime of the firm changing from one year to another i.e. want to know
3(a) how many of firms in regime 0 in 2011 moved to regime 2 in 2012
3(b) how many of firms in regime 2 in 2011 moved to regime 0,1, 2 in 2014
3(c) how many of firms in regime 1 in 2011 move to regime 2 in 2013
(3d) I also want to know how is this transition matrix calculated in stata i.e. for calculating transition matrix from 2011 to 2014. Is stata taking the data only for 2011 and 2015 or it is taking data for 2011,2012, 2013, 2014 to calculate transition matrix from 2011 to 2014?
(3e) what is the role of delta in xtset while finding these transition matrices?

Example generated by -dataex-. To install: ssc install dataex
clear
input float(cocode Year regime)
1 2011 1
1 2012 1
1 2013 .
1 2014 1
1 2015 1
2 2011 2
2 2012 2
2 2013 0
2 2014 0
2 2015 1
3 2011 1
3 2012 1
3 2013 .
3 2014 2
3 2015 .
4 2011 1
4 2012 1
4 2013 1
4 2014 0
4 2015 2
5 2011 2
5 2012 2
5 2013 2
5 2014 0
5 2015 0
end

[Nick Cox]

On (i) I am at a loss to know what you seek here. xttrans2 from SSC is a 2002 hack but its help contains the text you cite as a precise description that answers your question.

WIthout addressing the rest of the post in detail I cite https://www.stata.com/meeting/boston...14_nichols.pdf as an impressive overvlew of the field. As I recall it focuses on more fruitful approaches.

[Jessica Thacker]

Dear Nick Cox ,
Thank you so much for your reply and sharing the PDF..

On (i), I would like to know the know the exact difference between xttrans and xttrans2. I have not been able to figure this out through the help files and in (ii) I would like to know what is rectangularized data

I would humbly like to request you to please address (iii) part of my question. I have gone through the PDF that has been attached to the post but it does not answer my question. I would be really grateful if you could help me with the same.

Warm regards,
Jessica

[Nick Cox]

I was addressing (i) only in detail without taking on your several other questions.

If the help doesn't explain clearly enough, the code for xttrans and xttrans2 can be viewed in your favourite text editor.

To repeat, xttrans2 was a hack in 2002 -- almost certainly in response to a Statalist question at the time, although the emails have long since disappeared -- adding some functionality to what xttrans. offered at the tome.

I doubt I have used it myself since and I've not kept track of what xttrans does now. In my Stata I see for xttrans

*! version 1.2.0 15apr2020

What I recall from Austin Nichols' presentation is that it is a lot easier to follow what is happening using lag operators.

[Jessica Thacker]

Thank you so much Nick Cox!
You are right the lag operators work much better for construction of transition probabilities.
And, interestingly for my data, the inconsistency that Austin pointed out in the presentation does not hold. Most likely, that is because I had used fillin command to fill the data with missing observations for the years for which data was unavailable!
Thank you again for your help.