command to check for overlapping interval?

[Trang Thu Vu]

Hello everyone,

I'd like to ask for your help in the following issue.

I'm using individual data and have the below table for their loss-aversion (lamda) interval in Hypothetical and Real task. In each task, an individual is in a specific lamda interval or in other word we know their lamda's min and max value in each situation.

My concern is I want to find out for each individual whether their lamda inverval in hypothetical task is overlapped with the lamda interval in real task.

Can you suggest me which command/way in Stata I should use?

Hypothetical task
Numbers of accepted options	r = 2.91	r = 1.96	r = 0.66	r = 0.31
0 (Reject all)	λ ≤ 0.12	λ ≤ 0.35	λ ≥ 1.45	λ ≥ 2.13
1	0.12 < λ ≤ 0.27	0.35 < λ ≤ 0.51	1.27 ≤ λ < 1.45	1.61 ≤ λ < 2.13
2	0.27 < λ ≤ 0.46	0.51 < λ ≤ 0.68	1.15 ≤ λ < 1.27	1.32 ≤ λ < 1.61
3	0.46 < λ ≤ 0.71	0.68 < λ ≤ 0.84	1.06 ≤ λ < 1.15	1.13 ≤ λ < 1.32
4	0.71 < λ ≤ 1	0.84 < λ ≤ 1	1 ≤ λ < 1.06	1 ≤ λ < 1.13
5	1 < λ ≤ 1.34	1 < λ ≤ 1.16	0.95 ≤ λ < 1	0.90 ≤ λ < 1
6 (Accept all)	λ > 1.34	λ > 1.16	λ < 0.95	λ < 0.90
Real task
	r = 2.91	r = 1.96	r = 0.66	r = 0.16	r= 0
Reject all	λ ≤ 0.03	λ ≤ 0.18	λ ≥ 1.84	λ ≥ 4.5	λ > 6
1	0.03 < λ ≤ 0.12	0.18 < λ ≤ 0.35	1.45 ≤ λ < 1.84	2.52 ≤ λ < 4.5	3 ≤ λ < 6
2	0.12 < λ ≤ 0.27	0.35 < λ ≤ 0.51	1.27 ≤ λ < 1.45	1.79 ≤ λ < 2.52	2≤ λ < 3
3	0.27 < λ ≤ 0.46	0.51 < λ ≤ 0.68	1.15 ≤ λ < 1.27	1.41 ≤ λ < 1.79	1.5 ≤ λ < 2
4	0.46 < λ ≤ 0.71	0.68 < λ ≤ 0.84	1.06 ≤ λ < 1.15	1.17 ≤ λ < 1.41	1.2 ≤ λ < 1.5
Accept all	λ > 0.71	λ > 0.84	λ < 1.06	λ < 1.17	λ < 1.20

Much appreciated!

[Clyde Schechter]

There is no command in Stata dedicated to this task, but it is not hard to code. If you show an example of your Stata data set, using the -dataex- command (of course!), specific advice can be given. If you are running version 16 or a fully updated version 15.1 or 14.2, -dataex- is already part of your official Stata installation. If not, run -ssc install dataex- to get it. Either way, run -help dataex- to read the simple instructions for using it. -dataex- will save you time; it is easier and quicker than typing out tables. It includes complete information about aspects of the data that are often critical to answering your question but cannot be seen from tabular displays or screenshots. It also makes it possible for those who want to help you to create a faithful representation of your example to try out their code, which in turn makes it more likely that their answer will actually work in your data.

[Trang Thu Vu]

Hi Clyde Schechter,

Thank you for your suggestion!

I use Stata 16 and follow your advice with -dataex- and copy the data

In which, lamda1h, lamda1e = lower bound of lamda for hypothetical and real task, respectively
lamda2h, lamda22 = upper bound of lamda for hypothetical and real task, respectively

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(lamda1h lamda2h lamda1e lamda2e)
   .  .12    .  .03
   .  .12  .71  .03
   .  .12  .71  .03
 .95    1 1.41 1.17
   .    . 1.15    .
   .    .  .03  .71
1.45    .    .    .
   1 1.06 1.84 1.41
1.06 1.15  .35 1.79
   1 1.13    .  1.5
   .   .9  1.5    .
   .   .9    .    .
 .51  .68    . 1.45
1.32 1.61  .84    3
1.13 1.32    .    2
   .   .9    .    .
   .    .    .  .46
   .  .12    .  .03
1.32 1.61  .84    3
   .    .    .    .
   .    .    .    .
   .  .12  .03  .03
   .    .    .    .
1.45    .    .    .
   .    .  .18    .
1.45    .    .    .
   .    .  .35    .
   1 1.13 1.84  1.5
2.13    .    .    .
   .    .    .    .
   .   .9    .    .
1.13 1.32  1.2    2
1.32 1.61    .    3
1.61 2.13    .    6
   .  .12    .  .03
2.13    .    .    .
   .    .    .    .
1.34    .    .    .
   .    .    .    .
 .12  .27    .  .35
 .12  .27  .03  .35
   .  .12  .84  .03
   1 1.13    .  1.5
2.13    .  .03    .
   .  .12 1.79  .03
1.45    .    .    .
   .   .9    .    .
1.16    .    .    .
1.32 1.61    .    3
   .  .12  .03  .03
   .    .  .51  .46
  .9    1 1.17  1.2
 .35  .51    . 1.84
   .    .  .35    .
 .12  .27  .18  .35
1.15 1.27    . 2.52
 .27  .46    .  .51
1.13 1.32    6    2
   1 1.06  1.5 1.41
   .  .12 1.84  .03
2.13    . 1.41    .
1.32 1.61  .12    3
  .9    1 1.41  1.2
   .    .    .    .
   .    .    .    .
 .95    1 1.45 1.17
   .    .    .    .
1.32 1.61  4.5    3
   .    .    .    .
   .   .9    3    .
   .    .    .    .
1.06 1.15  4.5 1.79
   .    .    .    .
   1 1.13 1.41  1.5
   .   .9    .    .
   .    .    .  .46
 .12  .27    .  .35
2.13    .    .    .
   1 1.13 1.17  1.5
1.45    .    .    .
   .    .    .    .
   1 1.16    . 1.06
1.27 1.45  .12  4.5
 .84    1    . 1.15
   1 1.13    .  1.5
1.61 2.13    .    6
 .71    1    .  .84
   .   .9    .    .
1.32 1.61 1.15    3
   .   .9    6    .
   1 1.06  4.5 1.41
   .   .9 1.15    .
   .   .9    .    .
   .   .9 1.17    .
1.13 1.32    .    2
   .   .9 1.06    .
1.32 1.61  .03    3
   .  .95    .    .
   .    .    .    .
   .    .    .    .
end

------------------ copy up to and including the previous line ------------------


Thank you,

[Clyde Schechter]

Sixteen of the 100 observations have invalid data because they show lamda2e < lamda1e, which cannot be possible when lamda2e is the upper bound and lamda1e is the lower bound.

Anyway, here is code that flags the invalid observations, and determines for the others whether the intervals overlap or not:

//  DEAL WITH MISSING VALUES
mvencode lamda1*, mv(`=c(minfloat)')
mvencode lamda2*, mv(`=c(maxfloat)')

//  IDENTIFY INVALID OBSERVATIONS
gen byte invalid_h = lamda1h > lamda2h
gen byte invalid_e = lamda1e > lamda2e

//  ASSESS OVERLAP
gen test1 = min(lamda2h, lamda2e)
gen test2 = max(lamda1e, lamda1h)
gen byte overlap = test1 >= test2 if !invalid_h & !invalid_e

//  RESTORE MISSING VALUES
mvdecode lamda1*, mv(`=c(minfloat)')
mvdecode lamda2*, mv(`=c(maxfloat)')

Note from the pedantry corner: the correct spelling of λ is lambda.

[Trang Thu Vu]

Thank you very much for sending me the code and pointing out my mistakes. That's great help.

I tried the code and it works well. My other wonders is that whether we can code to know the degree of overlap and of not overlap

For instance,

If overlap, totally overlap when lambda1h=lambda1e & lambda2h=lambda2e. For this I can code as it's straightforward.
mostly overlap: the two intervals are overlapped more than 50%
slightly overlap: the two intervals are overlapped less than 50% or 30%

If not overlap, whether the two intervals are close to each other?
whether they are very far away from each other?

For the other degrees, I still haven't figured out.

[Clyde Schechter]

mostly overlap: the two intervals are overlapped more than 50%

slightly overlap: the two intervals are overlapped less than 50% or 30%

50% (or 30%) of what? The interval between test2 and test1 is the intersection of the two intervals, test2 being its lower bound and test1 being its upper bound. You can compare the length of that interval to whatever it is you have in mind as the denominator of your percentage.

If not overlap, whether the two intervals are close to each other?

whether they are very far away from each other?

test2 - test1 measures the distance between the lower bound of the higher interval and the upper bound of the lower interval when there is no overlap. You would have to decide how big a distance is "far" and how small a distance is "close."

[Trang Thu Vu]

I got it. Thank you for your advice!

Can I have another question about coding?

When I create lambda1h, lambda2b, lambda1e, lambda2e, I need to write very long code. For example:

gen lambda1h=0.12 if h_safeoption==3&h_accepted==1
replace lambda1h=0.27 if h_safeoption==3&h_accepted==2
replace lambda1h=0.46 if h_safeoption==3&h_accepted==3
replace lambda1h=0.71 if h_safeoption==3&h_accepted==4
replace lambda1h=1 if h_safeoption==3&h_accepted==5
replace lambda1h=1.34 if h_safeoption==3&h_accepted==6

replace lambda1h=0.35 if h_safeoption==2&h_accepted==1
replace lambda1h=0.51 if h_safeoption==2&h_accepted==2
replace lambda1h=0.68 if h_safeoption==2&h_accepted==3
replace lambda1h=0.84 if h_safeoption==2&h_accepted==4
replace lambda1h=1 if h_safeoption==2&h_accepted==5
replace lambda1h=1.16 if h_safeoption==2&h_accepted==6

replace lambda1h=1.45 if h_safeoption==1&h_accepted==0
replace lambda1h=1.27 if h_safeoption==1&h_accepted==1
replace lambda1h=1.15 if h_safeoption==1&h_accepted==2
replace lambda1h=1.06 if h_safeoption==1&h_accepted==3
replace lambda1h=1 if h_safeoption==1&h_accepted==4
replace lambda1h=0.95 if h_safeoption==1&h_accepted==5

replace lambda1h=2.13 if h_safeoption==0&h_accepted==0
replace lambda1h=1.61 if h_safeoption==0&h_accepted==1
replace lambda1h=1.32 if h_safeoption==0&h_accepted==2
replace lambda1h=1.13 if h_safeoption==0&h_accepted==3
replace lambda1h=1 if h_safeoption==0&h_accepted==4
replace lambda1h=0.9 if h_safeoption==0&h_accepted==5

and I repeated it four times.

Are there any other shorter way to create the four lambdas?

I'd like to send the example of my data set as below.


----------------------- copy starting from the next line -----------------------

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(h_safeoption h_accepted e_safeoption e_accepted)
0  6 0  5
1  0 0  5
0  1 0  1
0  3 0  5
0  6 0  5
1  5 0  5
0  2 0  5
3  3 0  5
0  6 0  5
1  5 0  5
0  1 0  2
1  6 0  5
0  6 0  1
3  6 0  5
1  1 0  5
4  0 0  5
0  6 0  5
0  5 0  5
4  5 0  1
0  6 0  5
0  0 0  3
0  0 0  5
0  3 0  5
0  6 0  5
0  6 0  5
0  3 0  4
0  5 0  5
0  2 0  5
0  3 0  0
2  2 0  5
4  0 0  0
0  0 0  5
0  0 0  5
0  0 0  5
0  6 0  3
0  0 0  5
0  5 0  5
0  6 0  5
0  6 0  5
0  6 0  5
2  3 0  1
0 98 0  5
2  5 0  5
4 98 0  3
0  6 0  5
0  4 0  5
1  5 0  5
0  6 0  5
0  2 0  5
0  0 0  1
0  3 0  5
2  6 0  2
0  0 0  0
1  5 0  5
2  2 0  5
0  6 0  5
4  6 0  5
1  4 0  3
1  2 0  1
0  6 0  5
0  6 0  5
0  6 0  0
4  0 0  5
4  0 0  2
0  6 0  5
3  2 1  5
0  5 1  4
0  4 1  3
0  1 1  3
0  6 1  4
0  3 1  5
0  5 1  2
0  5 1  3
0  5 1  3
0  1 1  1
0  5 1  3
0  0 1 98
0  6 1  3
0  2 1  5
1  4 1  0
0  0 1  3
2 98 1  5
0  1 1  5
1  4 1  5
0  0 1  5
0  4 1  5
0  2 1  0
0  0 1  3
4  0 1  0
2  0 1  2
0  5 1  3
0  4 1  3
1  3 1  0
0  4 1  3
3  0 1  0
1  5 1  3
0  6 1  3
0  5 1  5
0  3 1  5
0  4 1  2
end

------------------ copy up to and including the previous line ------------------

[Clyde Schechter]

Well, as best I can tell the values of lambdah1 are "magic numbers" with no particular pattern (other than a pattern of increasing with both safeoption and accepted). So the alternative of just wrigin out some algebraic expression to calculate these values seems not to be possible. Since the coded is tedious to write out, I suggest instead creating a data set with variables h_safeoption, h_accepted, and lambda1h that consists of one observation for each combination of h_safeoption and h_accepted, and the corresponding values of lambda1h. Doing that still requires typing out all those values, but at least it spares you all the -replace- and -if- and other stuff required for code. Then you can merge that new data set with your original data to bring the lambda values in.

So, like this:

clear*
* Example generated by -dataex-. To install: ssc install dataex
clear
input float h_safeoption byte h_accepted float lambda1h
3 1  .12
3 2  .27
3 3  .46
3 4  .71
3 5    1
3 6 1.34
2 1  .35
2 2  .51
2 3  .68
2 4  .84
2 5    1
2 6 1.16
1 0 1.45
1 1 1.27
1 2 1.15
1 3 1.06
1 4    1
1 5  .95
0 0 2.13
0 1 1.61
0 2 1.32
0 3 1.13
0 4    1
0 5   .9
end
tempfile lambdas
save `lambdas'

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(h_safeoption h_accepted e_safeoption e_accepted)
0  6 0  5
1  0 0  5
0  1 0  1
0  3 0  5
0  6 0  5
1  5 0  5
0  2 0  5
3  3 0  5
0  6 0  5
1  5 0  5
0  1 0  2
1  6 0  5
0  6 0  1
3  6 0  5
1  1 0  5
4  0 0  5
0  6 0  5
0  5 0  5
4  5 0  1
0  6 0  5
0  0 0  3
0  0 0  5
0  3 0  5
0  6 0  5
0  6 0  5
0  3 0  4
0  5 0  5
0  2 0  5
0  3 0  0
2  2 0  5
4  0 0  0
0  0 0  5
0  0 0  5
0  0 0  5
0  6 0  3
0  0 0  5
0  5 0  5
0  6 0  5
0  6 0  5
0  6 0  5
2  3 0  1
0 98 0  5
2  5 0  5
4 98 0  3
0  6 0  5
0  4 0  5
1  5 0  5
0  6 0  5
0  2 0  5
0  0 0  1
0  3 0  5
2  6 0  2
0  0 0  0
1  5 0  5
2  2 0  5
0  6 0  5
4  6 0  5
1  4 0  3
1  2 0  1
0  6 0  5
0  6 0  5
0  6 0  0
4  0 0  5
4  0 0  2
0  6 0  5
3  2 1  5
0  5 1  4
0  4 1  3
0  1 1  3
0  6 1  4
0  3 1  5
0  5 1  2
0  5 1  3
0  5 1  3
0  1 1  1
0  5 1  3
0  0 1 98
0  6 1  3
0  2 1  5
1  4 1  0
0  0 1  3
2 98 1  5
0  1 1  5
1  4 1  5
0  0 1  5
0  4 1  5
0  2 1  0
0  0 1  3
4  0 1  0
2  0 1  2
0  5 1  3
0  4 1  3
1  3 1  0
0  4 1  3
3  0 1  0
1  5 1  3
0  6 1  3
0  5 1  5
0  3 1  5
0  4 1  2
end

merge m:1 h_safeoption h_accepted using `lambdas'

Notes:
1. I note that your code in #7 does not include commands to cover combinations of h_accepted = 6 with h_safeoption == 0 or 1, nor h_accepted = 0 with h_safeoption = 2 or 3. But these combinations do appear in your -dataex- example. Moreover, your -dataex- example includes observations with h_safeoption = 4, and some with h_accepted = 98, neither of which is covered (in combination with anything) by your replace code.

2. You can expand that new data set to include the other lambdas as well, as additional variables. If you do that, then, if you are running version 16, rather than -merge-ing in you can put it in a separate frame and set up separate linkds for h_safeoption h_accepted, e_safeoption e_connected and then -frget- the appropriate variables that way. If you are running an earlier version of Stata, then frames are not available to you, and using a single file to merge these in will require cloneing or renaming variables to get the merge keys right.

Finally, this question is off the topic of the thread. While it is easy to think of these threads as dialogs between a questioner and an answerer, they are, in fact, read by others who follow along or who come to the Forum from time to time searching topics. When the content of the thread veers off topic, this results in some material being impossible for those others to find, and other material that they are not interested in being thrust in their way and wasting their time. So it is important to keep threads on topic. This question should have been posted in a new thread. If there is any follow-up to it, please start a new thread (and include a link back to this one).

[Trang Thu Vu]

Thank you for your suggestion! This could be one way to create lambda.

I'm sorry I haven't mentioned how I create lambda. So the algebraic expression to calculate these values you mentioned may come from that source.

I created another thread at this link: https://www.statalist.org/forums/for...ate-a-variable