[Clyde Schechter]

In the dataex, JCS is always the extreme.

There are two -dataex- examples in this thread, at #5 and #10. For the one at #5, JCS is always missing. For the one at #10, JCS is always between pres and senate. So, so far, we have no -dataex- examples where JCS is an extreme value.

Now, re-running my code with #5, I notice that it does not handle the missing JCS properly. The following revision of the code does:

rename (house JCS senate pres) ideol=
gen `c(obs_t)' obs_no = _n

reshape long ideol, i(obs_no) j(inst) string
drop if missing(ideol)

by obs_no (ideol), sort: egen JCS_order = max(cond(inst == "JCS", _n, .))
by obs_no (ideol): gen SOPconstraint = 0 if !inlist(JCS_order, 1, _N) & !missing(JCS_order)
by obs_no (ideol): replace SOPconstraint = ideol[2]-ideol[1] if JCS_order == 1
by obs_no (ideol): replace SOPconstraint = ideol[_N]-ideol[_N-1] if JCS_order == _N
drop JCS_order

reshape wide
rename ideol* *

The graph in #13 does show some cases where JCS is an extreme value, and in #14 I suggested that O.P. post some of those data. He has not responded to that. So I extracted the data from his graph and ran my code, as corrected in this post, on his data. And it appears to produce correct results; in any case, it certainly does not produce all zeroes.

. clear*

. graph use "C:\Users\clyde\Downloads\SOP constraint depicted.gph"

.
. //      EXTRACT THE DATA FROM THE GRAPH
. serset dir

  0.  18196 observations on 5 variables
      pres JCS senate house year

. serset set 0

. serset use

. summ

    Variable |        Obs        Mean    Std. dev.       Min        Max
-------------+---------------------------------------------------------
        pres |     17,550    .0299904    .4914045      -.504       .692
         JCS |     16,290    .0416405     .148949   -.296214   .2412361
      senate |     18,195   -.0269929    .0613419      -.131       .134
       house |     18,195   -.0467056    .0552665      -.136       .117
        year |     18,195    1967.219    19.84449       1925       1996

.
. //      CALCULATE THE SOP_constraint
. rename (house JCS senate pres) ideol=

. gen `c(obs_t)' obs_no = _n

.
. reshape long ideol, i(obs_no) j(inst) string
(j = JCS house pres senate)

Data                               Wide   ->   Long
-----------------------------------------------------------------------------
Number of observations           18,196   ->   72,784      
Number of variables                   6   ->   4           
j variable (4 values)                     ->   inst
xij variables:
    ideolJCS ideolhouse ... ideolsenate   ->   ideol
-----------------------------------------------------------------------------

. drop if missing(ideol)
(2,554 observations deleted)

.
. by obs_no (ideol), sort: egen JCS_order = max(cond(inst == "JCS", _n, .))
(5,070 missing values generated)

. by obs_no (ideol): gen SOPconstraint = 0 if !inlist(JCS_order, 1, _N) & !missing(JCS_order)
(29,310 missing values generated)

. by obs_no (ideol): replace SOPconstraint = ideol[2]-ideol[1] if JCS_order == 1
(1320 real changes made)

. by obs_no (ideol): replace SOPconstraint = ideol[_N]-ideol[_N-1] if JCS_order == _N
(22920 real changes made)

. drop JCS_order

.
. reshape wide
(j = JCS house pres senate)

Data                               Long   ->   Wide
-----------------------------------------------------------------------------
Number of observations           70,230   ->   18,195      
Number of variables                   5   ->   7           
j variable (4 values)              inst   ->   (dropped)
xij variables:
                                  ideol   ->   ideolJCS ideolhouse ... ideolsenate
-----------------------------------------------------------------------------

. rename ideol* *

.
.
. summ SOPconstraint

    Variable |        Obs        Mean    Std. dev.       Min        Max
-------------+---------------------------------------------------------
SOPconstra~t |     16,290    .0368904    .0599033          0   .2276687

. count if missing(JCS)
  1,905

. count if missing(SOPconstraint)
  1,905

.

[Gordon Ballingrud]

If JCS is at the extreme, SOP should not register as 0. it should register as the absolute value of JCS and the nearest pivot. Looks like it's often the House. SOP should register as 0 only if JCS is not at one of the extremes. That means the Court is within the Pareto set, and should be unconstrained.

EDIT: whoa, thanks Clyde! just saw this new code. I will get to work on it.

[Clyde Schechter]

You can verify the code is working correctly (at least as far as whether SOP turns out to be zero) by running the following afterwards:

egen low = rowmin(JCS-senate)
egen high = rowmax(JCS-senate)
assert (SOPconstraint !=0) == (missing(JCS) | inlist(JCS, low, high))

These three commands verify that SOPconstraint != 0 if and only if JCS is either missing or is the lowest or highest of the values among JCS, pres, house, and senate. The code is, in this respect, working properly if the -assert- command produces no output. If the code is getting this wrong, you will get an error message from the -assert- command. If that happens, post back with a -dataex- example that reproduces the error.

[Gordon Ballingrud]

Wow--I can't thank you enough. It's extremely generous of you and others (but especially Clyde) to take out so much time to help me out with this issue which I suppose I could have brute-forced if it really came down to it. But I'm learning a lot, and if I can ever repay the favor, please let me know so I can.

I'll get to work on it, and get back to y'all soon.

[Gordon Ballingrud]

Ok so does the same code above still work if this is the relevant data instead of what I showed before? I realized I had to recode the JCS variable since the dataset I'm using codes dates by the year, and the Court's terms span two different years. There's one median per term, but that means possibly two medians per year since the scores change at least a little each time a new term begins (October; ends mid-late June).

Anyhoo, I'm still having trouble once I get to:

. by obs_no (ideol), sort: egen JCS_order = max(cond(inst == "JCS", _n, .))
(5070 missing values generated)

. by obs_no (ideol): gen SOPconstraint = 0 if !inlist(JCS_order, 1, _N) & !missing(JCS
> _order)
(5,070 missing values generated)

. by obs_no (ideol): replace SOPconstraint = ideol[2]-ideol[1] if JCS_order == 1
(0 real changes made)

. by obs_no (ideol): replace SOPconstraint = ideol[_N]-ideol[_N-1] if JCS_order == _N
(0 real changes made)

. su SOP

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
SOPconstra~t |     65,160           0           0          0          0

. tab SOP, m

SOPconstrai |
         nt |      Freq.     Percent        Cum.
------------+-----------------------------------
          0 |     65,160       92.78       92.78
          . |      5,070        7.22      100.00
------------+-----------------------------------
      Total |     70,230      100.00

The commands where no changes are being made is of course what concerns me.

[Clyde Schechter]

So using the data scraped from the graph in #20, I get the following results:

. clear*

.
. graph use "C:\Users\clyde\Downloads\SOP constraint depicted(1).gph"

.
. serset dir

  0.  18196 observations on 5 variables
      pres JCS senate house year

.
. serset set 0

.
. serset use, clear

.
. rename (house JCS senate pres) ideol=

. gen `c(obs_t)' obs_no = _n

.
. reshape long ideol, i(obs_no) j(inst) string
(j = JCS house pres senate)

Data                               Wide   ->   Long
-----------------------------------------------------------------------------
Number of observations           18,196   ->   72,784      
Number of variables                   6   ->   4           
j variable (4 values)                     ->   inst
xij variables:
    ideolJCS ideolhouse ... ideolsenate   ->   ideol
-----------------------------------------------------------------------------

. drop if missing(ideol)
(2,554 observations deleted)

.
. by obs_no (ideol), sort: egen JCS_order = max(cond(inst == "JCS", _n, .))
(5,070 missing values generated)

. by obs_no (ideol): gen SOPconstraint = 0 if !inlist(JCS_order, 1, _N) & !missing(JCS_order)
(29,310 missing values generated)

. by obs_no (ideol): replace SOPconstraint = ideol[2]-ideol[1] if JCS_order == 1
(1320 real changes made)

. by obs_no (ideol): replace SOPconstraint = ideol[_N]-ideol[_N-1] if JCS_order == _N
(22920 real changes made)

. drop JCS_order

.
. reshape wide
(j = JCS house pres senate)

Data                               Long   ->   Wide
-----------------------------------------------------------------------------
Number of observations           70,230   ->   18,195      
Number of variables                   5   ->   7           
j variable (4 values)              inst   ->   (dropped)
xij variables:
                                  ideol   ->   ideolJCS ideolhouse ... ideolsenate
-----------------------------------------------------------------------------

. rename ideol* *

.
. summ SOPconstraint

    Variable |        Obs        Mean    Std. dev.       Min        Max
-------------+---------------------------------------------------------
SOPconstra~t |     16,290    .0368904    .0599033          0   .2276687

Note: These are the same results as were obtained earlier in #16. It is most likely that the graph sent in #20 is the same graph you sent previously. If you intended to send a different graph, something went wrong in that process.

As you can see, I am not able to reproduce your results. Either the data you are working with is not the data shown in the graph, or you are doing something else earlier in the code that is either corrupting the data or breaking the code. But I can't help you without access to the actual data that reproduces the problems you are encountering.

All I can tell you at this point is that those data are not the data that is in the graph you have sent.

[Gordon Ballingrud]

Ah, ok. Let me see about getting that graph corrected:

I tried to overwrite the original graph and maybe did something wrong there. This one should be the correct one.



SOP constraint depicted new 9-4.gph

I'm heartened that the code is working on your end, Clyde! That suggests that something is just wrong over here. And if this graph/data works for you, then I'll know for sure it's just something on my end. Which I hope won't be too hard to work out.

I'll give it another run after I get home from class. About to head out now. Thank you once again for your help.

[Clyde Schechter]

. clear*

. graph use "C:\Users\clyde\Downloads\SOP constraint depicted new 9-4.gph"

. serset dir

  0.  18196 observations on 5 variables
      house JCS senate pres year

. serset set 0

. serset use, clear

.
. rename (house JCS senate pres) ideol=

. gen `c(obs_t)' obs_no = _n

.
. reshape long ideol, i(obs_no) j(inst) string
(j = JCS house pres senate)

Data                               Wide   ->   Long
-----------------------------------------------------------------------------
Number of observations           18,196   ->   72,784      
Number of variables                   6   ->   4           
j variable (4 values)                     ->   inst
xij variables:
    ideolJCS ideolhouse ... ideolsenate   ->   ideol
-----------------------------------------------------------------------------

. drop if missing(ideol)
(2,554 observations deleted)

.
. by obs_no (ideol), sort: egen JCS_order = max(cond(inst == "JCS", _n, .))
(5,070 missing values generated)

. by obs_no (ideol): gen SOPconstraint = 0 if !inlist(JCS_order, 1, _N) & !missing(JCS_order)
(34,914 missing values generated)

. by obs_no (ideol): replace SOPconstraint = ideol[2]-ideol[1] if JCS_order == 1
(2040 real changes made)

. by obs_no (ideol): replace SOPconstraint = ideol[_N]-ideol[_N-1] if JCS_order == _N
(27804 real changes made)

. drop JCS_order

.
. reshape wide
(j = JCS house pres senate)

Data                               Long   ->   Wide
-----------------------------------------------------------------------------
Number of observations           70,230   ->   18,195      
Number of variables                   5   ->   7           
j variable (4 values)              inst   ->   (dropped)
xij variables:
                                  ideol   ->   ideolJCS ideolhouse ... ideolsenate
-----------------------------------------------------------------------------

. rename ideol* *

.
. summ SOPconstraint

    Variable |        Obs        Mean    Std. dev.       Min        Max
-------------+---------------------------------------------------------
SOPconstra~t |     16,290    .1007198    .1457275          0   .4420117

. egen low = rowmin(JCS-senate)

. egen high = rowmax(JCS-senate)

. assert (SOPconstraint !=0) == (missing(JCS) | inlist(JCS, low, high))

.

So, I do not reproduce the problem you are having with this new data.

[Gordon Ballingrud]

Hey Clyde--which version of Stata are you using? I suspect it would not matter, but I use SE 15.1, and apparently the newest version is available at our IT department.

Still having the same problem on my end, but that must be unique to me. My next plan is to try these commands on a different computer since something is clearly wrong with something on my device over here, whether it's with the Stata version or the laptop itself, or some such.

I'll make a final note when I get it to work at last, but in the meantime, thank you again, everyone, but especially Clyde, for your time, effort, and generous assistance.

[Clyde Schechter]

I am using Stata version 18.5 MP4 (StataNow) running under Windows 10 64-bit. It shouldn't matter. All of the commands in question go back to very early versions of Stata, way beyond version 15.1.

Good luck!