Between Group Baseline Difference in Randomized Control Trial?

[Brad Anderson]

I know many of you are much smarter than I.. I'm analyzing data for an RCT with relatively small n (110). The outcome is mean drinks / day. At baseline, the between group difference in mean drinks per day is over 2. Not quite statistically significant, but ... There are pretty extreme outliers, but we went back to the original data and the numbers are what the numbers are. My view is that this so confounds the interpretation of intervention effects. In a fixed effects, or within subject change framework, the higher mean in one group at baseline essentially means there is more room to reduce over time. In an ANCOVA type framework, any between group differences are largely moot (unless they are much larger and/or are different in terms of direction at follow-up than at baseline. The investigators are not especially happy with my conclusion. Any insight or wisdom would be much appreciated.

[Roman Mostazir]

If the randomisation was properly carried out, any baseline difference is due to chance factor. In your ancova model the baseline scores are presumably used as a predictor/independent variable, in which case it does not matter if you have some dispersed values in your predicted variable. The derivative will still be consistent. Check out the distribution of residulats after ancova which should be normally distributed and if not, then there are concerns that your model might be influenced by some influential observations.

[Brad Anderson]

The distribution is very skewed with about 15% - 20% of observations at the lower limit of 0. I should have been more clear by what I meant by ANCOVA type framework. My thought after exploring the data was to use a mixed effects generalized linear model with log link, Poisson family error distribution, and robust standard errors. Covariates would include mean drinks per day at baseline, indicator variables for month of assessment, and treatment condition. Between group differences are pretty consistent over time, but that includes differences at baseline. E.g., around 2 drinks per day difference at all time points. Even though the coefficient for intervention is statistically significant, I find it difficult to look at the data and conclude there was an actual intervention effect that wasn't an artifact of the baseline difference. I generated the intervention schedule. I assume this was adhered to in the field and that the baseline difference is simply attributable to chance. And I think an analysis of within subject change is potentially problematic because one arm has more distance to the floor. If I didn't know the baseline score, I'd conclude that there was a significant intervention effect of around 2 drinks per day. Knowing the baseline difference, that doesn't seem accurate. Thanks again for your help.

[Carlo Lazzaro]

Brad:
with a RCT you should have a statistical plan already detailed in the study protocol before seeing the data.
I'm probably missing out on something, but I find weird that you have to choose a statistical approach after having explored the study results.

[Roman Mostazir]

Please note it is stated in the FAQ section to provide a fragment of data sample so that forum users know about your data structure. Now I am confused what is your ananlysis. Yous said 'mixed' design which is different than an ancova design. Mixed will produce time related change (Group:A: time0-time1) - (Group:B: time0-time1) as coefficient while ancova produces group differences after outcome is adjusted for baseline covariates. ancova is often applied when the primary outcome is just after the baseline measurement i.e. you have two time points only. If the primary outcome assessment is after more timepoints/measurements, then 'mixed'. also I am confused that if you have used mixed, then you have your data in long-shape which already has baseline (time0) drinking score in the oucome and yet baseline drinking score is being added as a covariate too. You could have been a bit more elaborative on your study design i.e. # of measurements, your data structure and the precise model (Stata commands) you used which would have saved time meaningfully for everyone. My sincer apologies if I can't commit to further assistance here as it is a work from home time and my home computer is sluggish to process my office work. But hopefully someone else will.