Summing Observations Together

[Francesco de Luca]

Hello everyone and thank you for accepting me to the forum. I am working on an assignment based on the data set attached (ps3-prio.dta) and I've been asked to discuss the distribution of the dependent variable (conflict) across two independent variables, year and country. Specifically, I am supposed to use histograms to discuss the distribution and the prevalence of conflicts across countries and years. I understand that I need to use the egen command to group the number of observed conflicts by year and by country, but I am unsure about the syntax. I have the following:

egen conflictyear = total(conflict), by(year)
tab conflictyear
histogram conflictyear

egen conflictcountry = total(conflict), by(cname)
tab conflictcountry
histogram conflictcountry

Thank you for your kind help.

[Jared Greathouse]

Please post back your data using the dataex command.

[Francesco de Luca]

Example:

* Example generated by -dataex-. For more info, type help dataex
clear
input str32 cname str3 cabbr int(ccode year) byte conflict float(latitude longitude) double literacy float(refugees gdp trade) byte polity float conflictyear
"Guinea-Bissau"     "GNB" 404 1995 0 12.07582 -14.64071   44  9.642123  6.450583  3.846106  . 10
"Guinea-Bissau"     "GNB" 404 1996 0 12.07582 -14.64071 42.5  9.642123  6.531382  3.746209  5 12
"Guinea-Bissau"     "GNB" 404 1997 0 12.07582 -14.64071   41  9.680344  6.566634 4.1092315  5 15
"Guinea-Bissau"     "GNB" 404 1998 0 12.07582 -14.64071 39.5  8.794825  6.209523 3.9177375  5 12
"Guinea-Bissau"     "GNB" 404 1999 0 12.07582 -14.64071   38   8.86785  6.256979  4.214756  0 13
"Guinea-Bissau"     "GNB" 404 2000 0 12.07582 -14.64071   40  8.935904  6.300629 4.4230704  3 11
"Guinea-Bissau"     "GNB" 404 2001 0 12.07582 -14.64071   42   8.89563  6.272814 4.5169916  5 12
"Guinea-Bissau"     "GNB" 404 2002 0 12.07582 -14.64071   44  8.935904  6.168534 4.4088206  5 13
"Guinea-Bissau"     "GNB" 404 2003 0 12.07582 -14.64071   46  8.935904  6.131467  4.346779  5  8
"Guinea-Bissau"     "GNB" 404 2004 0 12.07582 -14.64071   48  8.927446  6.122295 4.3588443 -1  9
"Guinea-Bissau"     "GNB" 404 2005 0 12.07582 -14.64071   50  8.938006  6.126208 4.4826984 -1  6
"Guinea-Bissau"     "GNB" 404 2006 0 12.07582 -14.64071   52         .  6.114069  4.548179  6  9
"Guinea-Bissau"     "GNB" 404 2007 0 12.07582 -14.64071   54         .  6.111294 4.4887853  6 11
"Equatorial Guinea" "EQG" 411 1995 0   1.6366  10.47573   69         .  7.891347  5.071532  . 10
"Equatorial Guinea" "EQG" 411 1996 0   1.6366  10.47573   70         .  8.123163  5.472247 -5 12
"Equatorial Guinea" "EQG" 411 1997 0   1.6366  10.47573   71         .  8.636743  5.412487 -5 15
"Equatorial Guinea" "EQG" 411 1998 0   1.6366  10.47573   72         .  8.810854  5.617616 -5 12
"Equatorial Guinea" "EQG" 411 1999 0   1.6366  10.47573   73         .  9.133703         . -5 13
"Equatorial Guinea" "EQG" 411 2000 0   1.6366  10.47573   74         .  9.236445  5.214458 -5 11
"Equatorial Guinea" "EQG" 411 2001 0   1.6366  10.47573   75         .   9.69729   5.26421 -5 12
"Equatorial Guinea" "EQG" 411 2002 0   1.6366  10.47573   76         .  9.860788  5.014727 -5 13
"Equatorial Guinea" "EQG" 411 2003 0   1.6366  10.47573   77         .  9.960393  5.150142 -5  8
"Equatorial Guinea" "EQG" 411 2004 0   1.6366  10.47573   78         .  10.21585  5.050617 -5  9
"Equatorial Guinea" "EQG" 411 2005 0   1.6366  10.47573   79         . 10.258928  4.972439 -5  6
"Equatorial Guinea" "EQG" 411 2006 0   1.6366  10.47573   80         . 10.178164  4.974428 -5  9
"Equatorial Guinea" "EQG" 411 2007 0   1.6366  10.47573   81         . 10.272375  5.005351 -5 11
"Gambia"            "GAM" 420 1995 0 13.45759 -15.29753   28  8.794825   6.90684  4.803715  . 10
"Gambia"            "GAM" 420 1996 0 13.45759 -15.29753 29.5  8.839276  6.892273  4.704095 -7 12
"Gambia"            "GAM" 420 1997 0 13.45759 -15.29753   31   8.89563  6.903903 4.5516834 -6 15
"Gambia"            "GAM" 420 1998 0 13.45759 -15.29753 32.5    9.2399  6.902643  4.688322 -5 12
"Gambia"            "GAM" 420 1999 0 13.45759 -15.29753   34  9.752665  6.929786 4.5937424 -5 13
"Gambia"            "GAM" 420 2000 0 13.45759 -15.29753 35.5  9.392662  6.949355 4.6523676 -5 11
"Gambia"            "GAM" 420 2001 0 13.45759 -15.29753   37 8.9996195  6.972699  4.346289 -5 12
"Gambia"            "GAM" 420 2002 0 13.45759 -15.29753 38.5  9.400961  6.907568 4.5355697 -5 13
"Gambia"            "GAM" 420 2003 0 13.45759 -15.29753   40  8.922658  6.943626 4.5584383 -5  8
"Gambia"            "GAM" 420 2004 0 13.45759 -15.29753 41.5  8.901503  6.963243  4.697935 -5  9
"Gambia"            "GAM" 420 2005 0 13.45759 -15.29753   43  8.899731  6.982947  4.702318 -5  6
"Gambia"            "GAM" 420 2006 0 13.45759 -15.29753 44.5         .  7.017784         . -5  9
"Gambia"            "GAM" 420 2007 0 13.45759 -15.29753   46         .  7.059601         . -5 11
"Mali"              "MLI" 432 1995 0 17.32995 -3.310206   21  9.792556  6.636408 4.0491714  . 10
"Mali"              "MLI" 432 1996 0 17.32995 -3.310206   22  9.809176  6.641634 4.0215125  7 12
"Mali"              "MLI" 432 1997 0 17.32995 -3.310206   23  9.441452  6.680495 4.1411476  7 15
"Mali"              "MLI" 432 1998 0 17.32995 -3.310206   24  9.517825  6.712192 4.0948935  6 12
"Mali"              "MLI" 432 1999 0 17.32995 -3.310206   25  9.024011  6.749898  4.168809  6 13
"Mali"              "MLI" 432 2000 0 17.32995 -3.310206 26.5  9.035987  6.753193  4.192153  6 11
"Mali"              "MLI" 432 2001 0 17.32995 -3.310206   28  9.035987  6.838473  4.425612  6 12
"Mali"              "MLI" 432 2002 0 17.32995 -3.310206 29.5   9.11603  6.849625 4.2640643  6 13
"Mali"              "MLI" 432 2003 0 17.32995 -3.310206   31 9.2103405  6.891421 4.1554947  6  8
"Mali"              "MLI" 432 2004 0 17.32995 -3.310206 32.5  9.328656  6.882832 4.1453385  6  9
"Mali"              "MLI" 432 2005 0 17.32995 -3.310206   34 9.3266115  6.911569 4.1420093  6  6
"Mali"              "MLI" 432 2006 0 17.32995 -3.310206 35.5         .  6.932905 4.2814217  6  9
"Mali"              "MLI" 432 2007 1 17.32995 -3.310206   37         .  6.930415 4.1636415  6 11
"Senegal"           "SEN" 433 1995 1 14.45408 -14.23445   34 11.109458  7.177499 4.2223535  . 10
"Senegal"           "SEN" 433 1996 0 14.45408 -14.23445 34.5 11.082143  7.170778  4.090146 -1 12
"Senegal"           "SEN" 433 1997 1 14.45408 -14.23445   35  10.95431  7.175058  4.108691 -1 15
"Senegal"           "SEN" 433 1998 1 14.45408 -14.23445 35.5 11.015345   7.20599   4.12647 -1 12
"Senegal"           "SEN" 433 1999 1 14.45408 -14.23445   36  9.975808  7.241193 4.1448894 -1 13
"Senegal"           "SEN" 433 2000 1 14.45408 -14.23445   37  9.942708  7.246364  4.176568 -1 11
"Senegal"           "SEN" 433 2001 1 14.45408 -14.23445   38  9.937889  7.264857 4.1970606  8 12
"Senegal"           "SEN" 433 2002 0 14.45408 -14.23445   39  9.937889  7.245146 4.2123365  8 13
"Senegal"           "SEN" 433 2003 1 14.45408 -14.23445   40  9.937889  7.283522 4.1800466  8  8
"Senegal"           "SEN" 433 2004 0 14.45408 -14.23445   41  9.942901  7.314945  4.186819  8  9
"Senegal"           "SEN" 433 2005 0 14.45408 -14.23445   42  9.938469  7.344053  4.240637  8  6
"Senegal"           "SEN" 433 2006 0 14.45408 -14.23445   43         .  7.341596  4.236857  8  9
"Senegal"           "SEN" 433 2007 0 14.45408 -14.23445   44         .  7.360677  4.212361  8 11
"Benin"             "BEN" 434 1995 0 9.616513  2.552924   32  10.07744  6.963343  3.974093  . 10
"Benin"             "BEN" 434 1996 0 9.616513  2.552924   33  8.699514  6.985415  3.813612  6 12
"Benin"             "BEN" 434 1997 0 9.616513  2.552924   34  7.972466  7.014844  3.805648  6 15
"Benin"             "BEN" 434 1998 0 9.616513  2.552924   35  7.972466  7.029834    3.7975  6 12
"Benin"             "BEN" 434 1999 0 9.616513  2.552924   36  8.216088   7.04604  3.805982  6 13
"Benin"             "BEN" 434 2000 0 9.616513  2.552924 37.5   8.36637  7.071764 3.7680976  6 11
"Benin"             "BEN" 434 2001 0 9.616513  2.552924   39  8.476371  7.088844 3.7630656  6 12
"Benin"             "BEN" 434 2002 0 9.616513  2.552924 40.5  8.517193  7.100471  3.714261  6 13
"Benin"             "BEN" 434 2003 0 9.616513  2.552924   42  8.517193  7.106055 3.6939924  6  8
"Benin"             "BEN" 434 2004 0 9.616513  2.552924 43.5  8.476788  7.104145  3.673231  6  9
"Benin"             "BEN" 434 2005 0 9.616513  2.552924   45 10.318705  7.100876   3.67821  6  6
"Benin"             "BEN" 434 2006 0 9.616513  2.552924 46.5         .   7.10694         .  6  9
"Benin"             "BEN" 434 2007 0 9.616513  2.552924   48         .  7.122027         .  7 11
"Mauritania"        "MAA" 435 1995 0 20.27244 -10.07208   38 10.445812  7.392467 4.4067583  . 10
"Mauritania"        "MAA" 435 1996 0 20.27244 -10.07208 38.5  9.674074  7.421055 4.3781657 -6 12
"Mauritania"        "MAA" 435 1997 0 20.27244 -10.07208   39  8.922658  7.351555 4.2327604 -6 15
"Mauritania"        "MAA" 435 1998 0 20.27244 -10.07208 39.5 3.0445225  7.350483 4.6476912 -6 12
"Mauritania"        "MAA" 435 1999 0 20.27244 -10.07208   40  5.298317  7.386299 4.5729947 -6 13
"Mauritania"        "MAA" 435 2000 0 20.27244 -10.07208 40.5  5.991465  7.375618  4.791475 -6 11
"Mauritania"        "MAA" 435 2001 0 20.27244 -10.07208   41  5.991465   7.37489  4.460161 -6 12
"Mauritania"        "MAA" 435 2002 0 20.27244 -10.07208 41.5  5.991465  7.356487  4.494278 -6 13
"Mauritania"        "MAA" 435 2003 0 20.27244 -10.07208   42  6.214608  7.381836 4.4579096 -6  8
"Mauritania"        "MAA" 435 2004 0 20.27244 -10.07208 42.5  6.159095  7.403823  4.706018 -6  9
"Mauritania"        "MAA" 435 2005 0 20.27244 -10.07208   43  6.448889  7.429154  4.879776 -6  6
"Mauritania"        "MAA" 435 2006 0 20.27244 -10.07208 43.5         .  7.512984 4.7332935 -5  9
"Mauritania"        "MAA" 435 2007 0 20.27244 -10.07208   44         .  7.506712  4.808209 -3 11
"Niger"             "NIR" 436 1995 0 17.42163   9.57933   13  10.22557  6.382287  3.724877  . 10
"Niger"             "NIR" 436 1996 1 17.42163   9.57933 13.5  10.15813  6.380024 3.6990416  8 12
"Niger"             "NIR" 436 1997 2 17.42163   9.57933   14  8.909235  6.370976  3.699604 -6 15
"Niger"             "NIR" 436 1998 0 17.42163   9.57933 14.5  8.216088  6.433789  3.786287 -6 12
"Niger"             "NIR" 436 1999 0 17.42163   9.57933   15  5.991465   6.39189 3.6449964 -6 13
"Niger"             "NIR" 436 2000 0 17.42163   9.57933   16 4.6051702  6.341784  3.772628  5 11
"Niger"             "NIR" 436 2001 0 17.42163   9.57933   17 4.6051702  6.374844  3.726555  5 12
"Niger"             "NIR" 436 2002 0 17.42163   9.57933   18  5.703783  6.369106  3.671678  5 13
"Niger"             "NIR" 436 2003 0 17.42163   9.57933   19  5.703783  6.377076  3.753467  5  8
end

[Jared Greathouse]

Okay so what's the problem? You use egen's total function... does it not give you what you seek?


You write you want the distribution...... have you considered violin plots?

[Francesco de Luca]

Basically I want to plot two histograms: one that shows the total number (frequency) of conflicts on the vertical axis and the number of countries on the horizontal axis, and one that shows the total number (frequency) of conflicts on the vertical axis and the year number on the horizontal axis. Stata is summing the number of observations and not the frequency for every year and every country.

[Nick Cox]

Here's some technique. When a group of observations necessarily has the same value on that variable, and we want to see that value just once, a solution lies in the tag() function of egen -- fortuitously documented just above the total() function in its help.

 tag(varlist) [, missing]
            may not be combined with by.  It tags just one observation in each distinct group defined by
            varlist.  When all observations in a group have the same value for a summary variable calculated for
            the group, it will be sufficient to use just one value for many purposes.  The result will be 1 if
            the observation is tagged and never missing, and 0 otherwise.  Values for any observations excluded
            by either if or in are set to 0 (not missing).  Hence, if tag is the variable produced by egen tag =
            tag(varlist), the idiom if tag is always safe.  missing specifies that missing values of varlist may
            be included.

        total(exp) [, missing]                                                              (allows by varlist:)  
            creates a constant (within varlist) containing the sum of exp treating missing as 0.  If missing is
            specified and all values in exp are missing, newvar is set to missing.  Also see mean().

.  egen c_total = total(conflict), by(cname)

. egen y_total = total(conflict), by(year)

. egen c_tag = tag(cname)

. egen y_tag = tag(year)

.. list year y_total if y_tag, noobs

  +----------------+
  | year   y_total |
  |----------------|
  | 1995         1 |
  | 1996         1 |
  | 1997         3 |
  | 1998         1 |
  | 1999         1 |
  |----------------|
  | 2000         1 |
  | 2001         1 |
  | 2002         0 |
  | 2003         1 |
  | 2004         0 |
  |----------------|
  | 2005         0 |
  | 2006         0 |
  | 2007         1 |
  +----------------+

. list cname c_total if c_tag, noobs

  +-----------------------------+
  |             cname   c_total |
  |-----------------------------|
  |     Guinea-Bissau         0 |
  | Equatorial Guinea         0 |
  |            Gambia         0 |
  |              Mali         1 |
  |           Senegal         7 |
  |-----------------------------|
  |             Benin         0 |
  |        Mauritania         0 |
  |             Niger         3 |
  +-----------------------------+

I wouldn't use histograms here at all, but quantile plots, as you have at most hundreds of countries and tens of years, and the detail of very high values could and should be shown more effectively than being in a hard to see bin with a very small frequency.