Rangerun AND Loops - Variables within program not getting created

[Nilima Pisharody]

Explanation: I realize that rangerun itself emulates a loop structure. But I want to perform it for multiple variables. When I run my program, no new variables are created and I want to know if there is a fatal flaw in my thinking. I also tried trace on and found out that the `thing' from inside the foreach loop is not getting passed to the program three_mths.

My code looks almost exactly like this

sysuse sp500.dta, clear
drop volume change

global things high low open close

clear programs
program three_mths
    egen `1'_3m = pctile(`1'), p(25)
    label variable `1'_3m "Top 25 - last 3 months"
end

set trace on
foreach thing in $things {
    rangerun three_mths, use(`thing') inter(date -90 -1)
}

I realize that you might ask me to unwrap the loop and call rangerun multiple times for each variable high, low, open and close separately. However, the actual dataset I am working with has more than 20 variables. And I need to do the percentile thing for 3, 6, 9, and 12 months - which is why getting this small code correct is so important.

I am also open to other suggestions which might make this easier and/or correct.
Thank you so much, in advance.

[Oscar Ozfidan]

I think I see the potential problem. Your high is -1 of date so the current obs is not included in the rolling sample and that is why it is not generating the variable. Try changing it to 0 and control the sample by "if _n <_N"

[Nilima Pisharody]

I think I see the potential problem. Your high is -1 of date so the current obs is not included in the rolling sample and that is why it is not generating the variable. Try changing it to 0 and control the sample by "if _n <_N"

I am not entirely sure of this. Because there are 2 points to note:
1. I do not want to take the current observation (I want the averages to start from my first lag)
2. More importantly, it works with the high date of -1 without the loop. I ran that first and got no errors.

[William Lisowski]

In your program, the local `1' is undefined because rangerun does not pass arguments to your program. Perhaps the following suggestions will show you a useful direction.

capture program drop three_mths
program three_mths
    egen returned = pctile(`1'), p(25)
end

foreach thing in $things {
    rangerun three_mths, use(`thing') inter(date -90 -1)
    rename returned `thing'_3m
    label variable `thing'_3m "Top 25 - last 3 months"
}

[Nick Cox]

rangerun is a community-contributed command ,from SSC, as you are asked to explain (FAQ Advice #12).

The 25th percentile is the lower quartile -- in what sense that ties up with your wording "Top 25" I don't understand. But I would never fire up egen again and again to get a percentile. There is too much slow code there.

I can't see your bug, but again I would not assign the same variable label to four different variables.

You could use rangestat (also SSC): there is a detailed example in the help. This seems to work:

sysuse sp500.dta, clear
drop volume change

* ssc inst moremata needed for -mm_quantile()-
mata:  
mata clear
real rowvector loq(real colvector X) {
    return(mm_quantile(X, 1,  0.25))
}
end

foreach thing in high low open close {
    rangestat (loq) `thing', interval(date -90 -1)
    rename loq1 `thing'_p25
}

[Oscar Ozfidan]

This seems to work

capture program drop myprog
program myprog
foreach thing in high low open close {
egen `thing'_3m = pctile(`thing'), p(25)
label variable `thing'_3m "Top 25 - last 3 months"
}
end
rangerun myprog, interval(date -90 -1) verbose

[Attaullah Shah]

asrol has a built-in option for finding percentiles. This can simplify the code to one line. I am using version 5.0 of asrol that has a flexible window.

net install asrol, from(http://fintechprofessor.com) replace

foreach v in high low open close {
    asrol  `v', gen(`v'_pc25)  window(date -91 -1) stat(median) perc(.25)
}

list open high high_pc25 low_pc25 in 50/100

    
     +-------------------------------------------+
     |    open      high   high_pc25    low_pc25 |
     |-------------------------------------------|
 50. | 1197.66   1197.66   1267.6899     1253.16 |
 51. | 1166.71   1182.04     1267.42     1252.26 |
 52. | 1173.56   1173.56      1266.5     1241.71 |
 53. | 1150.53    1173.5     1265.62     1241.56 |
 54. | 1170.81   1180.56     1264.74     1241.41 |
     |-------------------------------------------|
 55. | 1142.62   1149.39     1263.86     1234.04 |
 56. | 1122.14   1124.27     1263.47     1229.65 |
 57. | 1117.58   1141.83    1261.705    1229.035 |
 58. | 1139.83   1160.02   1259.9399     1228.42 |
 59. | 1152.69   1183.35   1252.8199     1228.33 |
     |-------------------------------------------|
 60. | 1182.17   1182.17     1251.01     1219.74 |
 61. | 1153.29   1161.69     1246.78     1217.59 |
 62. | 1147.95    1162.8     1242.55   1215.4399 |
 63. | 1160.33   1169.51     1241.36      1214.5 |
 64. | 1145.87   1145.87     1233.42     1176.78 |
     |-------------------------------------------|
 65. | 1106.46    1117.5     1197.83      1171.5 |
 66. | 1103.25   1151.47     1197.66     1166.71 |
 67. | 1151.44   1151.44     1183.35     1155.35 |
 68. | 1128.43   1146.13     1182.17     1150.96 |
 69. | 1137.59   1173.92     1182.04     1148.64 |
     |-------------------------------------------|
 70. | 1168.38   1182.24   1180.5601     1147.83 |
 71. | 1165.89   1183.51   1180.5601     1147.83 |
 72. |  1183.5   1184.64   1180.5601     1147.83 |
 73. | 1179.68   1192.25   1180.5601     1147.83 |
 74. | 1191.81   1248.42   1180.5601     1147.83 |
     |-------------------------------------------|
 75. | 1238.16   1253.71   1180.5601     1147.83 |
 76. |  1253.7    1253.7   1180.5601     1147.83 |
 77. | 1242.98   1242.98   1180.5601     1147.83 |
 78. | 1224.36   1233.54   1180.5601     1147.83 |
 79. | 1209.47   1232.36   1180.5601     1147.83 |
     |-------------------------------------------|
 80. | 1228.75    1248.3   1180.5601     1147.83 |
 81. | 1234.52   1253.07   1180.5601     1147.83 |
 82. | 1253.05    1269.3   1180.5601     1147.83 |
 83. | 1249.46   1266.47   1180.5601     1147.83 |
 84. | 1266.44   1272.93   1180.5601     1147.83 |
     |-------------------------------------------|
 85. | 1267.43   1267.43   1180.5601     1147.83 |
 86. | 1248.58   1267.51   1180.5601     1147.83 |
 87. | 1266.61      1270   1180.5601     1147.83 |
 88. | 1266.71   1267.01   1180.5601     1147.83 |
 89. |  1261.2   1261.65   1180.5601     1147.83 |
     |-------------------------------------------|
 90. | 1255.54   1268.14   1180.5601     1147.83 |
 91. | 1255.18   1259.84   1180.5601     1147.83 |
 92. | 1245.67   1249.68   1180.5601     1147.83 |
 93. | 1248.92   1257.45   1180.5601     1147.83 |
 94. | 1249.44   1286.39   1180.5601     1147.83 |
     |-------------------------------------------|
 95. | 1284.99   1296.48   1180.5601     1147.83 |
 96. | 1288.49   1292.06   1180.5601     1147.83 |
 97. | 1291.96   1312.95   1180.5601     1147.83 |
 98. | 1312.83   1315.93   1180.5601     1147.83 |
 99. | 1309.38   1309.38   1180.5601     1147.83 |
     |-------------------------------------------|
100. | 1289.05   1295.04   1180.5601     1147.83 |
     +-------------------------------------------+

[Nilima Pisharody]

Attaullah Shah This is actually very succinct. Thanks so much!

[Aditya Kashikar]

I ran into a similar issue in an unrelated project and just created a new variable to deal with it. All variables in your dataset are passed as arguments to program by default. "Use" in rangerun limits the access to the variables; useful if you want to speed things up but unnecessary if you have few variables in your financial dataset

CODE:
sysuse sp500.dta
drop volume change
global things high low open close
clear programs
program three_mths
egen whatIneed = pctile(tempy), p(25)
end
foreach thing in $things {
gen tempy=`thing'
rangerun three_mths, inter(date -90 -1)
label variable whatIneed "Top 25 - last 3 months"
rename whatIneed `thing'_3m
drop tempy
}

Hope this helps.