Kolmogorov-Smirnov test

[harry smith]

Hi all, another question. The previous advice has been so helpful! I ran an experiment on risk preferences: people make 10 decisions. Each decision is between two options (a safe option and a risky option), coded as 1 and 0 respectively. Additionally I ran an experiment on rolling a standard six sided die (it measures honesty).

Half the subjects completed both experiments in a control group. The other half completed the same experiments under slightly different conditions (treatment group). I have plotted both of them and found that the treatment makes no difference. Although I need a p-value to show this.

I have decided to use the Kolmogorov-Smirnov test.

Thus my command is: ksmirnov totalsafe, by(treatment)

totalsafe here being the number of safe choices in the risk-preferences experiment as explained above.

I have seen that in the past has recommended against the KS test. I have 2 questions here:

1) Is the above command find to satisfy the need for a p-value? (I am using it because the previous literature seems to use it)
2) Are there any other tests that you would recommend?

Thanks again, the help has been amazing!

[harry smith]

I am (politely bumping) this, if anyone can offer some guidance, I would be extremely grateful. Perhaps adding some extra information such as a relatively small sample size (I have between 50-70 observations). Additionally, I ask the question because when I searched through the forum, people have been critical of the KS test, which is why I was wondering if there may be a more suitable alternative.

[Bruce Weaver]

Hello Harry. Have you considered fracreg with the proportion of Safe responses as the DV? From the help:

Description

    fracreg fits a fractional response model for a dependent variable that is greater
    than or equal to 0 and less than or equal to 1.  It uses a probit, logit, or
    heteroskedastic probit model for the conditional mean. These models are often used
    for outcomes such as rates, proportions, and fractional data.

[harry smith]

Hi Bruce, thanks for the response! I had never heard about that, so will look into it. I have already ran OLS, and ordered logit models.

The logic behind trying to use the KS test was to try and demonstrate that the treatment had no effect, and so the distributions of the data are similar (I can very clearly through the aid of a visual plot).

[harry smith]

Last bump (apologies if this is annoying). If there is any additional information/clarification needed I can do so.

[Nick Cox]

Policy on bumping is explicit at https://www.statalist.org/forums/help#adviceextras -- and yes; you've bumped enough, thanks!

[David Kaplan]

2) You could also try distcomp which spreads power into the tails better than ksmirnov and also shows you the particular range(s) of values [if any] where the two ECDFs statistically differ; see
https://faculty.missouri.edu/kapland....html#distcomp
for a link to the Stata Journal article and installation notes. In particular, Section 4.3 of the article shows an example with the "gift exchange" experiment data from Gneezy and List (2006).

[David Kaplan]

Updated link for my prior post (university shut down our web server...):
https://kaplandm.github.io/#distcomp