Alternative to statsby

[Henry Strawforrd]

I am calculating the Spearman correlation between two variables and the respective over a large number of groups.

statsby r=r(rho) obs=r(N) p=r(p), by(group) clear: spearman wfe_time' ffe_time'

Since the data is large, I am having issues that statsby takes forever and then stata freezes, possibly due to memory.

What are more efficient ways to accomplish this (possibly also making sure that if Stata freezes, I have at least some results)?

[Rich Goldstein]

you might want to take a look at the user-written -runby- command; use -search- to find and install

[Nick Cox]

I make no claims about speed, but here is another way to do it with rangestat from SSC.

sysuse auto, clear 

gen touse = !missing(mpg, weight)

bysort touse foreign : egen rank1 = rank(mpg)

bysort touse foreign : egen rank2 = rank(weight)

rangestat (corr) rank? , interval(foreign 0 0) by(touse)

tabdisp foreign if touse, c(corr*)

------------------------------------------------------
Car       |
origin    |            corr_nobs                corr_x
----------+-------------------------------------------
 Domestic |                   52            -.87226376
  Foreign |                   22            -.76221199
------------------------------------------------------

To get a P-value is more work, but you could write your own program and feed it to rangerun (SSC).