Ordinal outcome data analysis methods

[Cathy Qi]

Hi, please could I have your opinions and advice on the following:

I have been given a study to analyse. The outcome measure is a questionnaire that outputs 3 possible scores: low risk /medium risk /high risk.

I want to compare the scores between two groups: active vs placebo.

The sample size is very small: 24 vs 20 so I think ordinal logistic regression would be inappropriate. I think a non-parametric method would be more suitable.

What do you think is the best way to analyse this data that would take into account the ordered nature of the outcome?

Many thanks in advance

Cathy

[Joseph Coveney]

What do you think is the best way to analyse this data that would take into account the ordered nature of the outcome?

Unless you know from theory that the conditional distributions grossly deviate from proportional odds, go for ordered logistic regression. A regression table gives you more to talk about than the output of a Mann-Whitney Wilcoxon test.

.ÿversionÿ15.1

.ÿ
.ÿclearÿ*

.ÿsetÿseedÿ`=strreverse("1452664")'

.ÿ
.ÿquietlyÿsetÿobsÿ`=24+20'

.ÿgenerateÿbyteÿtrtÿ=ÿ_nÿ>ÿ24

.ÿquietlyÿgenerateÿbyteÿresponseÿ=ÿ.

.ÿ
.ÿprogramÿdefineÿsimem,ÿrclass
ÿÿ1.ÿÿÿÿÿÿÿÿÿversionÿ15.1
ÿÿ2.ÿÿÿÿÿÿÿÿÿsyntax
ÿÿ3.ÿ
.ÿÿÿÿÿÿÿÿÿquietlyÿreplaceÿresponseÿ=ÿruniformint(1,ÿ3)
ÿÿ4.ÿ
.ÿÿÿÿÿÿÿÿÿologitÿresponseÿi.trt
ÿÿ5.ÿÿÿÿÿÿÿÿÿtestÿ1.trt
ÿÿ6.ÿÿÿÿÿÿÿÿÿtempnameÿol
ÿÿ7.ÿÿÿÿÿÿÿÿÿscalarÿdefineÿ`ol'ÿ=ÿr(p)
ÿÿ8.ÿ
.ÿÿÿÿÿÿÿÿÿranksumÿresponse,ÿby(trt)
ÿÿ9.ÿÿÿÿÿÿÿÿÿreturnÿscalarÿmwÿ=ÿ2ÿ*ÿnormal(-abs(r(z)))
ÿ10.ÿÿÿÿÿÿÿÿÿreturnÿscalarÿolÿ=ÿ`ol'
ÿ11.ÿend

.ÿ
.ÿsimulateÿmwÿ=ÿr(mw)ÿolÿ=ÿr(ol),ÿreps(10000)ÿnodots:ÿsimem

ÿÿÿÿÿÿcommand:ÿÿsimem
ÿÿÿÿÿÿÿÿÿÿÿmw:ÿÿr(mw)
ÿÿÿÿÿÿÿÿÿÿÿol:ÿÿr(ol)


.ÿ
.ÿforeachÿvarÿofÿvarlistÿmwÿolÿ{
ÿÿ2.ÿÿÿÿÿÿÿÿÿgenerateÿbyteÿp`var'ÿ=ÿ`var'ÿ<ÿ0.05
ÿÿ3.ÿ}

.ÿ
.ÿsummarizeÿp*

ÿÿÿÿVariableÿ|ÿÿÿÿÿÿÿÿObsÿÿÿÿÿÿÿÿMeanÿÿÿÿStd.ÿDev.ÿÿÿÿÿÿÿMinÿÿÿÿÿÿÿÿMax
-------------+---------------------------------------------------------
ÿÿÿÿÿÿÿÿÿpmwÿ|ÿÿÿÿÿ10,000ÿÿÿÿÿÿÿ.0478ÿÿÿÿ.2133535ÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿ1
ÿÿÿÿÿÿÿÿÿpolÿ|ÿÿÿÿÿ10,000ÿÿÿÿÿÿÿ.0497ÿÿÿÿ.2173353ÿÿÿÿÿÿÿÿÿÿ0ÿÿÿÿÿÿÿÿÿÿ1

.ÿ
.ÿexit

endÿofÿdo-file


.

search gologit2

[Nick Cox]

Cathy UK: Please use your full real name.

https://www.statalist.org/forums/help#realnames

https://www.statalist.org/forums/help#adviceextras #3

[Cathy Qi]

Hi Nick

Apologies, I have made a request to change my user name.

Bw
Cathy

[Joseph Coveney]

what . . . would generate this?

An expression containing a couple of rbinomial() functions. This kind of question has come up on the list before. You might want to search through the archives.

For another scale on the questionnaire (risk of having behavioural disorder), the data looks like this (see below). Would you still advise ologit/WMW or to forgo statistical analysis altogether?

My advice is to make your analysis decisions prior to gathering the data, and document them in your study's protocol or statistical analysis plan. This includes any decision tree about selecting the analytical method to use.

What do you see in those data there that raises the prospect of forgoing analysis altogether for your study's outcome of interest?

[Cathy Qi]

An expression containing a couple of rbinomial() functions. This kind of question has come up on the list before. You might want to search through the archives.

Thanks, I will look through the archives. I assume for now, something like uniform() and assigning values 1/2/3 at specific cutoffs would also do the job?

My advice is to make your analysis decisions prior to gathering the data, and document them in your study's protocol or statistical analysis plan. This includes any decision tree about selecting the analytical method to use.

What do you see in those data there that raises the prospect of forgoing analysis altogether for your study's outcome of interest?

The SAP specifies to use ordinal logistic regression (adjusting for 5 covariates) unless there is insufficient data then a chi-squared test will be used, so it was left a bit vague, I guess this was because the original sample size was 54 and a high proportion of missing data was anticipated (actual observed n=44). I reported the results of the ordinal logistic models as pre-specified but was advised to remove them, report the chi-sq, and if possible, look into more appropriate non-parametric methods that would take into account the ordered outcome due to the sparsity of the data, but my advisors were not entirely sure themselves how small a sample/no. events would be deemed “insufficient data” for this method and the reasons why fitting the model would be inappropriate, they expressed they would feel uncomfortable fitting the model to so little data. (As for forging the analysis, on some scales on the questionnaire, everyone scored low risk, we wondered if it would be appropriate to forgo analysing this questionnaire altogether.)