Acreg regression with big data set

[Michael Schuster]

Hi community,

I have a geocoded (5x5km grid cells) panel data set which ranges from 2000 to 2015 with 5 years gap (meaning that I have the years 2000, 2005, 2010, 2015, see example) over all of Sub-Saharan Africa. I have about 4 million observations. I run the rather new regression command -acreg- GDP growth on HIV prevalence or male circumcision as an instrument, respectively. As I am using Stata/BE 17.0 on a Mac this regression takes forever. I am thinking about splitting the dataset in any way (maybe by African regions), yet I would like to know the overall effect on Africa. Is there anyway I could split the dataset, get the output and combine the output again, to get to know the overall effect in Africa? Another solution would be to take a lot of subsamples and run the regressions a lot of times. Any suggestions are welcome!

grid_id longitude latitude year au_region sub_saharan country iso31c ccode province iso32c pcode growth_gdp_ppp_total hiv_mean_prev mc_mean_prev
865800 34,3125 7,770833 2000 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .0811733 .1089163
865800 34,3125 7,770833 2005 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .4857559 .0558064 .1015968
865800 34,3125 7,770833 2010 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .5976818 .0492746 .121218
865800 34,3125 7,770833 2015 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .4217044 .0534075 .1377779
865801 34,35416667 7,770833 2000 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .0823907 .1319166
865801 34,35416667 7,770833 2005 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .4581345 .0578024 .127765
865801 34,35416667 7,770833 2010 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .5700278 .0494164 .1537717
865801 34,35416667 7,770833 2015 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .4161915 .0541627 .1778983
865802 34,395833 7,770833 2000 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .0855318 .1681238
865802 34,395833 7,770833 2005 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .4581345 .0585013 .1644206
865802 34,395833 7,770833 2010 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .5700278 .0517092 .1791989
865802 34,395833 7,770833 2015 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .4161915 .0544366 .2127961
865803 34,4375 7,770833 2000 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .0872959 .1969659
865803 34,4375 7,770833 2005 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .4225288 .0602377 .1857397
865803 34,4375 7,770833 2010 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .534306 .0514357 .2268204
865803 34,4375 7,770833 2015 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .408924 .0549143 .2538433
865804 34,479167 7,770833 2000 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .0896141 .2184768
865804 34,479167 7,770833 2005 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .4225288 .0614122 .2230376
865804 34,479167 7,770833 2010 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .534306 .0528303 .2641923
865804 34,479167 7,770833 2015 East Africa 1 Ethiopia ETH 231 Gambela Peoples ET.GA 990 .408924 .0557831 .2732673

Thank you!

[Michael Schuster]

This is maybe better:

 * Example generated by -dataex-. For more info, type help dataex clear input long grid_id double(longitude latitude) float(year growth_GDP_PPP_total) double(HIV_mean_prev MC_mean_prev) float ln_gdp_ppp_total 612551 2.72916666678866 13.4791666663955 2000         .  .0113952420651913 .990811169147491 16.076998 612551 2.72916666678866 13.4791666663955 2005  .1706244 .00958700943738222 .994386434555054 16.234533 612551 2.72916666678866 13.4791666663955 2010 .22794993 .00588105153292418         .9955161  16.43988 612551 2.72916666678866 13.4791666663955 2015  .1681127 .00376865570433438         .9969429  16.59527 612552 2.77083333345551 13.4791666663955 2000         .  .0114019885659218 .990886151790619 15.395984 612552 2.77083333345551 13.4791666663955 2005 .15884513 .00973280053585768 .993543684482574 15.543407 612552 2.77083333345551 13.4791666663955 2010  .2160491 .00575962476432323         .9960363 15.739015 612552 2.77083333345551 13.4791666663955 2015  .1654788 .00376936281099916         .9968488 15.892147 612553 2.81250000012236 13.4791666663955 2000         .  .0111839063465595 .990696012973785 15.395984 612553 2.81250000012236 13.4791666663955 2005 .15884513 .00968935340642929 .994269490242004 15.543407 612553 2.81250000012236 13.4791666663955 2010  .2160491 .00581180397421122         .9959334 15.739015 612553 2.81250000012236 13.4791666663955 2015  .1654788 .00384826818481088         .9973352 15.892147 612554  2.8541666667892 13.4791666663955 2000         .  .0112042371183634 .991451442241669 14.863758 612554  2.8541666667892 13.4791666663955 2005 .20897754  .0095862802118063 .994564950466156 15.053534 612554  2.8541666667892 13.4791666663955 2010  .2674411 .00588257005438209         .9956774 15.290533 612554  2.8541666667892 13.4791666663955 2015 .17675897 .00391481164842844         .9969919 15.453298 612555 2.89583333345605 13.4791666663955 2000         .  .0114256078377366 .991257309913635 14.863758 612555 2.89583333345605 13.4791666663955 2005 .20897754 .00964953191578388 .994801461696625 15.053534 612555 2.89583333345605 13.4791666663955 2010  .2674411 .00580572756007314          .996235 15.290533 612555 2.89583333345605 13.4791666663955 2015 .17675897 .00385410059243441         .9970776 15.453298 612556  2.9375000001229 13.4791666663955 2000         .  .0112883280962706 .990532875061035 15.847743 612556  2.9375000001229 13.4791666663955 2005  .2267739 .00957189407199621 .994763970375061  16.05213 612556  2.9375000001229 13.4791666663955 2010 .28537965 .00596617860719562         .9965305 16.303185 612556  2.9375000001229 13.4791666663955 2015  .1805613 .00390501879155636         .9974594 16.469175 612557 2.97916666678975 13.4791666663955 2000         .  .0113649414852262 .991434395313263 15.847743 612557 2.97916666678975 13.4791666663955 2005  .2267739 .00968039501458406 .994557976722717  16.05213 612557 2.97916666678975 13.4791666663955 2010 .28537965 .00592478085309267          .995773 16.303185 612557 2.97916666678975 13.4791666663955 2015  .1805613 .00395362172275782         .9971524 16.469175 612558 3.02083333345659 13.4791666663955 2000         .  .0111322328448296 .991210341453552 15.373527 612558 3.02083333345659 13.4791666663955 2005 .21816477 .00974737480282784 .995118200778961 15.570872 612558 3.02083333345659 13.4791666663955 2010 .27670395 .00586639903485775         .9961905 15.815154 612558 3.02083333345659 13.4791666663955 2015 .17872886 .00384415732696652         .9976268  15.97959 612559 3.06250000012344 13.4791666663955 2000         .  .0109965046867728 .992109060287476 15.373527 612559 3.06250000012344 13.4791666663955 2005 .21816477 .00943726394325495  .99527645111084 15.570872 612559 3.06250000012344 13.4791666663955 2010 .27670395 .00583592336624861         .9966778 15.815154 612559 3.06250000012344 13.4791666663955 2015 .17872886 .00401432812213898         .9973729  15.97959 612560 3.10416666679029 13.4791666663955 2000         .  .0113323573023081 .992041945457458  14.89746 612560 3.10416666679029 13.4791666663955 2005  .2095828 .00946436263620853 .995438098907471 15.087736 612560 3.10416666679029 13.4791666663955 2010 .26805162 .00599796092137694         .9966905 15.325218 612560 3.10416666679029 13.4791666663955 2015 .17688897 .00396168138831854         .9971074 15.488092 612561 3.14583333345714 13.4791666663955 2000         .  .0110401883721352 .991976678371429  14.89746 612561 3.14583333345714 13.4791666663955 2005  .2095828 .00958679988980293 .995556831359863 15.087736 612561 3.14583333345714 13.4791666663955 2010 .26805162 .00583096733316779         .9964032 15.325218 612561 3.14583333345714 13.4791666663955 2015 .17688897 .00387062761001289         .9977303 15.488092 612562 3.18750000012398 13.4791666663955 2000         .  .0109043167904019 .991712152957916 14.707047 612562 3.18750000012398 13.4791666663955 2005  .2061671 .00945729669183493 .995767056941986 14.894495 612562 3.18750000012398 13.4791666663955 2010 .26460683 .00590855488553643         .9961565 15.129256 612562 3.18750000012398 13.4791666663955 2015 .17615303 .00406615389510989         .9978445 15.291505 612563 3.22916666679083 13.4791666663955 2000         .  .0111725609749556 .991860210895538 14.707047 612563 3.22916666679083 13.4791666663955 2005  .2061671  .0094987777993083 .995502114295959 14.894495 612563 3.22916666679083 13.4791666663955 2010 .26460683 .00594787485897541         .9967039 15.129256 612563 3.22916666679083 13.4791666663955 2015 .17615303 .00397224863991141         .9976168 15.291505 612564 3.27083333345768 13.4791666663955 2000         .  .0111121172085404 .991329789161682  14.57839 612564 3.27083333345768 13.4791666663955 2005  .2038649 .00957683473825455 .995276927947998 14.763928 612564 3.27083333345768 13.4791666663955 2010 .26228437 .00598770519718528         .9968782  14.99685 612564 3.27083333345768 13.4791666663955 2015  .1756559 .00399381387978792         .9975035 15.158677 612565 3.31250000012453 13.4791666663955 2000         .  .0114873237907887 .992097437381744  14.57839 612565 3.31250000012453 13.4791666663955 2005  .2038649 .00946289673447609 .995153963565826 14.763928 612565 3.31250000012453 13.4791666663955 2010 .26228437 .00585305970162153         .9963712  14.99685 612565 3.31250000012453 13.4791666663955 2015  .1756559 .00393765699118376         .9975535 15.158677 612566 3.35416666679138 13.4791666663955 2000         .  .0113830491900444 .991927325725555 14.350263 612566 3.35416666679138 13.4791666663955 2005  .1997931 .00933026615530252  .99556028842926 14.532412 612566 3.35416666679138 13.4791666663955 2010  .2581767 .00596584426239133         .9969015 14.762075 612566 3.35416666679138 13.4791666663955 2015  .1747739 .00400689383968711         .9977459  14.92315 612567 3.39583333345822 13.4791666663955 2000         .  .0112893683835864 .991742551326752 14.350263 612567 3.39583333345822 13.4791666663955 2005  .1997931 .00948534719645977 .995655298233032 14.532412 612567 3.39583333345822 13.4791666663955 2010  .2581767 .00597272533923388          .996778 14.762075 612567 3.39583333345822 13.4791666663955 2015  .1747739 .00397419044747949         .9978979  14.92315 612568 3.43750000012507 13.4791666663955 2000         .  .0112604601308703 .991497755050659 14.434425 612568 3.43750000012507 13.4791666663955 2005 .20129374 .00944491103291512 .995439946651459 14.617825 612568 3.43750000012507 13.4791666663955 2010 .25969052 .00585258239880204         .9967705  14.84869 612568 3.43750000012507 13.4791666663955 2015  .1750993 .00398071948438883         .9977684 15.010043 612569 3.47916666679192 13.4791666663955 2000         .  .0110805658623576  .99097603559494 14.434425 612569 3.47916666679192 13.4791666663955 2005 .20129374 .00929000787436962 .995123147964478 14.617825 612569 3.47916666679192 13.4791666663955 2010 .25969052 .00608902750536799          .997037  14.84869 612569 3.47916666679192 13.4791666663955 2015  .1750993 .00406918488442898         .9978561 15.010043 612570 3.52083333345876 13.4791666663955 2000         .  .0112151755020022 .991125106811523  14.24318 612570 3.52083333345876 13.4791666663955 2005  .1978866 .00939759518951178  .99509209394455 14.423738 612570 3.52083333345876 13.4791666663955 2010 .25625297 .00595856970176101         .9969543 14.651872 612570 3.52083333345876 13.4791666663955 2015 .17435986 .00402311561629176         .9977231 14.812595 612571 3.56250000012561 13.4791666663955 2000         .  .0111126946285367 .990645706653595  14.24318 612571 3.56250000012561 13.4791666663955 2005  .1978866 .00936393439769745 .995209515094757 14.423738 612571 3.56250000012561 13.4791666663955 2010 .25625297 .00603470485657454         .9964469 14.651872 612571 3.56250000012561 13.4791666663955 2015 .17435986 .00407639611512423         .9980413 14.812595 612572 3.60416666679246 13.4791666663955 2000         .  .0110590206459165 .990896940231323 14.360893 612572 3.60416666679246 13.4791666663955 2005  .1999826 .00932643376290798 .995149374008179   14.5432 612572 3.60416666679246 13.4791666663955 2010 .25836778 .00612880475819111         .9970773 14.773016 612572 3.60416666679246 13.4791666663955 2015 .17481497 .00410562753677368          .997489 14.934126 612573 3.64583333345931 13.4791666663955 2000         .    .01135338190943 .991044461727142 14.360893 612573 3.64583333345931 13.4791666663955 2005  .1999826 .00932671315968037 .994888842105865   14.5432 612573 3.64583333345931 13.4791666663955 2010 .25836778 .00605473481118679         .9963985 14.773016 612573 3.64583333345931 13.4791666663955 2015 .17481497 .00424656225368381         .9981191 14.934126 612574 3.68750000012615 13.4791666663955 2000         .  .0113125629723072 .990260899066925 14.683136 612574 3.68750000012615 13.4791666663955 2005 .20573898 .00950930174440145 .994950354099274  14.87023 612574 3.68750000012615 13.4791666663955 2010 .26417488 .00610860949382186         .9966736  15.10465 612574 3.68750000012615 13.4791666663955 2015 .17606068 .00418301625177264         .9977734  15.26682 612575   3.729166666793 13.4791666663955 2000         .  .0114521058276296 .991335511207581 14.683136 612575   3.729166666793 13.4791666663955 2005 .20573898 .00961873307824135 .994603216648102  14.87023 612575   3.729166666793 13.4791666663955 2010 .26417488 .00611228682100773         .9970282  15.10465 612575   3.729166666793 13.4791666663955 2015 .17606068 .00424239179119468         .9978874  15.26682 end

After that I regress as following:

acreg growth_GDP_PPP_total ln_gdp_ppp_total (HIV_mean_prev = MC_mean_prev), id(grid_id) time(year) lagcut(15)spatial latitude(latitude) longitude(latitude) dist(40)

Any suggestions on that?