outcome and baseline adjustment in observational studies: a view on this controversy from the STATALIST community

[Gianfranco Di Gennaro]

Dear all,

I recently posted about my concerns regarding the correct analysis of an observational study where I aimed to assess the effect of follow-up length on a group measured before and after the intervention.

Thanks again to Joseph Coveney , Erik Ruzek and Rich Goldstein for their precious contribution.

I still have concerns, and I see there is considerable neverending controversy in the literature about how to handle baseline measurements in observational studies.

Therefore I would like to have an opinion from this community where most people have much more expertise and experience than me. Above all I would like to understand if I have reasoned correctly in my years of work.


As an example of non-randomized study, let's take the typical scenario where I have a continuous outcome measured at baseline and post-exposure, and with an unexposed control group.


Question #1
Should the outcome be the change from baseline or the post-intervention value?

My opinion is that change from baseline is essentially a regression to the group-specific mean, and should be avoided. Is this correct?

***

Question #2
Should baseline values be used as a covariate for adjustment in the model?

I believe this second question deserves more attention than most researchers give it.
In most cases, baseline measurements are taken after exposure, which means they are often collinear with the group variable and therefore should not be included in the model. Is my reasoning correct?
However, if the cause-effect logic behind the observed phenomenon suggests it, the baseline could be considered as a mediator, and possibly analyzed as such. Furthermore, including baseline values would substantially alter the hypothesis addressed by the model (see Lord's paradox).


I would appreciate your thoughts on these two questions.

Kind regards, thanks for your time.
Gianfranco

[daniel klein]

Perhaps I have not read the previous conversation as thoroughly as I should have; I hope this is still helpful.

First, I believe you are conflating multiple issues here, e.g., conceptual causal models with (parametric) estimation strategies (and even specific estimators). Explicitly disentangling these aspects might help.

That said, I don't fully understand your questions either.

Question #1
Should the outcome be the change from baseline or the post-intervention value?

Are you asking whether we should restrict our comparison to post-treatment outcomes between the groups, essentially ignoring within-group differences? If so, the answer almost certainly has to be no. A difference-in-difference approach, i.e., comparing the pre-post treatment differences (1st difference) between groups (second difference), requires much weaker assumptions for identifying causal effects.

Question #2
Should baseline values be used as a covariate for adjustment in the model?

[...]
In most cases, baseline measurements are taken after exposure, which means they are often collinear with the group variable

I don't follow this. The term "baseline" implies measurement before treatment. Also, how does time of measurement affect (increase?) correlation (collinearity) with the group indicator? Could you clarify?

However, if the cause-effect logic behind the observed phenomenon suggests it, the baseline could be considered as a mediator, and possibly analyzed as such. Furthermore, including baseline values would substantially alter the hypothesis addressed by the model (see Lord's paradox).

The baseline could also be considered a confounder. Either way, the "cause-effect logic" (i.e., the causal model) cannot be empirically tested (Edit: That is to say, it includes untestable assumptions, which should be made explicit). In my limited understanding, the key to solving Lord's paradox is the distinction between conceptually identifying causal effects using causal models and estimating causal effects from data using econometrics; see my initial comment.

[Erik Ruzek]

There is a lot of literature on this topic, much of it in the experimental case. However, a recent paper by Lüdtke & Robitzsch (2023) does a great job going through the implications of both types of models in observational data.