Intersection of Stata and Hardware

[SJW]

I am looking to build a new workstation. I use Stata regularly, with the most intensive uses being rolling regressions over a panel of say N = 8000 and T = 2000, amounting to 4+ GB data sets. I was looking at an i7 based system (4930k), and asked the advice of some folks over at a hardware forum, who keep harping on the fact that I should build an intel Xeon based system. I am wondering if it makes a difference in my use case with Stata, as this is my primary concern? If I SHOULD use Xeon, can someone please explain to me why. Thanks in advance for your help.

[Joe Canner]

I don't have any insights into the difference between i7 and Xeon on a single user system, but there are two other things that may make more of a difference:

• With 4+ GB data sets, your most important purchase will be memory. Obviously you need to have at least that much, but it would probably be good to have at least 2-3 times that much to allow for overhead, other programs, OS needs, etc. Even in a system with adequate memory, Stata tends to slow down as the memory gets full.
• Using Stata/MP on a system with multiple processors is probably more important than the particular processor.

[SJW]

Joe, thanks for your helpful comments! These were my thoughts exactly. I planned on at least 32GB of memory with the option to expand to 64, but I was don't know as much about the differences in CPU options. As I said before, I don't see how it would make a difference beyond a slight difference in stability and potentially longevity, but I was getting nervous.

Regarding the TYPE of RAM, the hardware forum strongly urged using ECC RAM, which MIGHT be the reason for using a Xeon processor, as i7 doesn't have support for ECC, but does this really make a difference for a single user system? Thanks again!

[Robert Wells]

I don't have any insights into the difference between i7 and Xeon on a single user system, but there are two other things that may make more of a difference:

• With 4+ GB data sets, your most important purchase will be memory. Obviously you need to have at least that much, but it would probably be good to have at least 2-3 times that much to allow for overhead, other programs, OS needs, etc. Even in a system with adequate memory, Stata tends to slow down as the memory gets full.
• Using Stata/MP on a system with multiple processors is probably more important than the particular processor.

This is very true advice in my experience, memory is the most common limitation. Speed can be made up with by simply waiting longer. If speed is an issue, a colleague of mine once built a machine with a solid state hard drive for both hold stata, the data to be used and if possible the OS. This made things considerably faster, especially with datasets of the size you refer to.

[Bert Lloyd]

This is very true advice in my experience, memory is the most common limitation. Speed can be made up with by simply waiting longer. If speed is an issue, a colleague of mine once built a machine with a solid state hard drive for both hold stata, the data to be used and if possible the OS. This made things considerably faster, especially with datasets of the size you refer to.

Could you expand on why this would speed things up beyond initial startup and loading times? My (probably incorrect) understanding was that Stata "lived" in the computer's RAM - thus the struggles with very large datasets - so why would the hard drive make a difference?

[Joe Canner]

Bert,

I can't speak to Robert's experience, but in my experience the time required to initially load a large data set into memory can be a significant issue, especially if you are working with a lot of different data sets.

In addition, many Stata commands use temporary disk storage for various purposes, in which case a faster hard drive can be a big help.

Regards,
Joe