Stata 17 is now available

[Alan Riley (StataCorp)]

Stata 17 is here! You can read all about it on stata.com/new-in-stata.

Stata 17 has 29 major new features:

• Tables
• Bayesian econometrics
• Faster Stata
• Difference-in-differences (DID) and DDD models
• Interval-censored Cox model
• Multivariate meta-analysis
• Bayesian VAR models
• Treatment-effects lasso estimation
• Bayesian multilevel models: Nonlinear, joint, SEM-like, and more
• Galbraith plots
• Leave-one-out meta-analysis
• Bayesian longitudinal/panel-data models
• Panel-data multinomial logit model
• Zero-inflated ordered logit model
• Nonparametric tests for trend
• Bayesian dynamic forecasting
• Bayesian IRF and FEVD analysis
• BIC for lasso penalty selection
• Lasso for clustered data
• Bayesian linear and nonlinear DSGE models
• Do-file Editor enhancements
• New functions for dates and times
• Intel Math Kernel Library (MKL)
• Stata on Apple Silicon
• JDBC
• Java integration
• H2O integration
• PyStata
• Jupyter Notebook with Stata

I thought Stata 16 would be difficult to top, but I hope you find we were up to the challenge. Stata 17 has something exciting for everyone. Multiple somethings.

I'm posting a little more about this release on the Stata Blog. You can also visit stata.com/new-in-stata for full details about each of the above new features.

[Brian Poi]

Congratulations to the StataCorp team! For those who haven't experienced it, it can be hard to describe just how much work and stress goes into getting a new release out the door.

I'm most excited about the Intel MKL usage and the speed-ups it will bring as well as the other speed-ups mentioned.

[Sergio Correia]

Hi Alan,

Congratulations! One question: can you tell us a bit how exactly did you made collapse faster? Besides MKL and sort speedup, is there something else that changed in the algorithm?

Thanks!

[Alan Riley (StataCorp)]

Thank you, Brian Poi!

I got to do the fun part -- announcing the release. The StataCorp development, IT, marketing, production, and technical services teams did an amazing job getting this release out the door.

Since this is Statalist, I can share a little bit of insider thoughts with you all. You probably know we like to put out our releases roughly every two years. It has been roughly 22 months since the last release, meaning we had two less months than usual to work on all the features you now see. And, because of COVID-19, we have been working remotely for the last 13 months, which has required major changes to the way we put together and put out a new version of Stata. I'm incredibly proud of how well our staff adapted to that as well as what they accomplished.

A couple of things that aren't mentioned in our highlights that you might run across:

1) Stata/IC has been renamed to Stata/BE, for "Basic Edition". "IC" was a leftover from "Intercooled Stata".
2) Stata/SE is still Stata/SE, but the SE now stands for "Standard Edition" rather than "Special Edition". "Special Edition" made sense when it was our latest and greatest, but Stata/MP has been out for quite a while now.

I hope at least some of you find that your wish list (or wishes and grumbles) items have been addressed!

[Dave Airey]

Congratulations Alan and colleagues on making this release despite COVID-19. Fantastic accomplishment. I look forward to using the new tables feature among others!!

[Hua Peng (StataCorp)]

Sergio Correia , thanks a lot. I still remember your presentation of ftools in 2017 Stata conference Baltimore. Both ftools and Mauricio Caceres' wonderful gtools gave us a lot of motivation. This is definitely not the end rather the start of speeding up Stata's data management commands.

For the technical detail of making collapse faster, first, the most expensive part of the sorting is rearrange the data in in memory to the sorted order, this step is unnecessary for -collapse- on the original data, only on the collapsed data, which is one of the main changes. Second, most of the computation for various statistics are internalized and improved.

[Richard Williams]

This is great news. Congratulations.

For a complete list of all the changes made, I assume you can just check out

https://www.statalist.org/forums/for...t-for-stata-17

;-)

Thanks to Covid, all purchase requests above $10 at Notre Dame must now be approved by the Dean, the Provost, and at least two executive vice-presidents. ;-) Assuming I can get approval I will be ordering soon.

[Sergiy Radyakin]

Congratulations!! The new release looks great! I am looking forwards to enjoy the improvements in the do-editor and take advantage of the expanded python and java integration. It is also pleasing to see the attention to construction of tables from within Stata.

Has the format of data changed at all compared to v16?

[Richard Williams]

I'm looking forward to trying the new table commands, but it is a little daunting when you see the Tables manual is 271 pages long! Hopefully the basics aren't too hard. A Stata video on it would be nice.

I did try the xtmlogit command. I notice that it has a fixed effects option. I compared it to the user-written femlogit command and, at least in the example I tried, the results were identical. I occasionally get inquiries about panel data mlogit so I suspect some people will be very happy to see xtmlogit added.

[Richard Williams]

Has the format of data changed at all compared to v16?

Here is what the help for save says:

Stata 14 through 17 have the same dataset format so long as the dataset has 32,767 variables or less. Since Stata/MP 15, Stata/MP has supported more than 32,767 variables and thus has a slightly different dataset format when there are that many variables. If you are using Stata 17 and want to save a dataset so that it may be read by someone using Stata 16 or Stata 15, simply use the save command; those older versions will be able to read it. If the dataset has more than 32,767 variables, it can be read by Stata/MP 16 and Stata/MP 15. If you want to save a dataset so that it may be read by someone using Stata 14, again simply use the save command; Stata 14 will be able to read it so long as it does not have more than 32,767 variables. Stata 14 supports at most 32,767 variables.

saveold saves the dataset currently in memory on disk under the name filename in previous .dta formats, namely, those for Stata 13, 12, or 11. If you are using Stata 17 and want to save a file so that it may be read by someone using an older version of Stata, use the saveold command.

[Ahmed Khan]

First of all, congratulations to all for getting the New update. It is great to see STATA putting effort even during covid and lockdowns. I just got my stata 17 and I see huge improvements. I am using AMD Ryzen 7 4800H with 40GbRam. On my stata 16.1 MP8 my data with 44.7 Million obs used to take 30 seconds to sort, now on stata 17 MP8 it is taking 16.1 seconds to sort. I also ran few other commands where I manipulate the data. Stata 16.1 used to take 8 minutes to complete that task now Stata 17 takes 3 minutes to complete similar tasks.

I am not sure how Intel will perform with the new "Intel Math Kernel Library (MKL)" update. However, I am very happy with the new update. Just wanted to thank you guys for the update.

[Joseph Coveney]

During previous installations, I remember being prompted to uninstall the prior release's installation before completing installation of the new one. That didn't happen this time and I didn't notice any mention of it in the online guidance. I take it that it's become moot.

I plan to uninstall the previous release, anyway, but now that I've got the new release installed, I trust that uninstalling the prior one won't mess up the new one's installation, right?

[Rich Goldstein]

thanks to all at Stata corp for both (1) what looks like a really good release and (2) for positively responding to some of my requests

[Richard Williams]

During previous installations, I remember being prompted to uninstall the prior release's installation before completing installation of the new one. That didn't happen this time and I didn't notice any mention of it in the online guidance. I take it that it's become moot.

I plan to uninstall the previous release, anyway, but now that I've got the new release installed, I trust that uninstalling the prior one won't mess up the new one's installation, right?

I don't remember ever being asked to uninstall but in any event I never do it. I have all versions of Stata from 7 on installed on my machine. The help files come in handy, because if a program uses version control you can check how the command worked way back when. It is also handy when I update my own programs. For example, my gologit2 program requires Stata 11.2. If I make some minor tweak to it, I don't want it to suddenly require some feature only available in Stata 17 if there is a way to program around it.

Stata is one of the few programs that makes it easy to copy from one computer to another. I can just copy the folder the program is in. At least I've been able to do that in the past. That is, I don't have to actually install the old version, I just have to copy the files. With most other programs I have to reinstall from scratch when I go to a new machine.

[Erik Ruzek]

First, thanks for all the work on the new release - it looks fantastic. I have one big question, however.

Why did StataCorp use Intel MKL instead of BLIS or OpenBLAS, which utilize the SSE3-SSE4 or AVX1/2 features to speed up mathematical operations on Intel and non-Intel processors? If MKL doesn't detect a genuineIntel processor, it defaults to SSE1 on AMD or ARM (as applicable). Can StataCorp give us BLIS/OpenBLAS support in lieu of or in addition to MKL? There was a bit of a dust-up on this issue with MATLAB, including proposed workarounds. Thanks.