2x2 Mixed Factorial Design - Command

[Marvin Aliaga]

Let me give you a quick background of my design.

My main task is to "test" if staff members who participated in the counseling sessions had fewer "Use of Force (UOF)" incidents in the 6 moths after the counseling session. I have a data set with the number of UOF incidents for the staff member in the 6 months before (pre) and after the counseling (post). I also have a control group (staff who didn't participate in the counseling sessions).

Variables
group 1= experimental 0=control
UOF pre= continuous
UOF post= continuous


Other characteristics included: groups were not randomized. They are very similar in their demographic but differ in the pre UOF variable. Experimental group has a higher UOF in pre. There are 345 observations per group. The UOF pre = 2.8 post 1.7 for both group.

I know that my data is a count and that it violates some assumptions. However, this project is not very rigorous. I just want to have a general idea if the staff who received the counseling (experimental group) reduced the number of UOF in a greater rate that staff in the control group. I just want to see if the counseling is more effective in reducing UOF incidents than no counseling at all.

What is the command for a 2x2 Mixed Factorial ANOVA? Does my data need to be in long format? I red that you dont need to reshape your data long if you only have two factors per variable (group and time), is this correct?

I also try to look for count models for pre and post test with group variable but didn't fin anything.

Thank you so much!
Marvin

[Roman Mostazir]

I won't go into the fact of design issues you raised as you are aware of it. Mechanistical suggestion to your problem is a model with a count distribution (poisson distribution) as you have an outcome with count variable. Type - help poisson- for further insights about models with count outcome. Since you have two measures baseline and 6 month followup, comparing two groups (experimental vs. control) at 6-month adjusting for the baseline line score will be appropriate. This type of model is called ANCOVA (if you have other covariates in the model) model adjusted for baseline score. This will give you the mean difference in incidence rate between the groups after the outcome score is adjusted for any variance accountable for baseline score. Below is an example with hypothetical data:

*Example*

//uof0 is the baseline uof score and uof1 is the score after 6 months.
// grp=1(experiental group); grp=0(control group)
 

list in 1/10, noobs clean

    id   grp   uof0   uof1  
     1     0      8      9  
     2     0      5      9  
     3     0     10      7  
     4     0      9     10  
     5     1      9      2  
     6     1      5      2  
     7     0      9      9  
     8     0      8      5  
     9     0      6      9  
    10     1      8      6  

// Regression model adjusted with baseline score

 poisson uof1 i.grp uof0, irr

Iteration 0:   log likelihood = -2046.1222  
Iteration 1:   log likelihood =  -2046.122  

Poisson regression                              Number of obs     =      1,000
                                                LR chi2(2)        =     755.41
                                                Prob > chi2       =     0.0000
Log likelihood =  -2046.122                     Pseudo R2         =     0.1558

------------------------------------------------------------------------------
        uof1 |        IRR   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       1.grp |   .4642543   .0134886   -26.41   0.000     .4385558    .4914587
        uof0 |   .9970643   .0078186    -0.37   0.708     .9818572    1.012507
       _cons |   7.707599   .4677876    33.65   0.000     6.843184    8.681205
------------------------------------------------------------------------------

//Experimental group is showing 54% lower incidence rate than control group.
But I would not interpret the p'values as your data is not randomly collected.

[Marvin Aliaga]

Hi Roman Mostazir ,

Thank you so much! This help me a lot. Below you can find my actual result.

. poisson PostUOF c.PreUOF c.Group,irr

Iteration 0:   log likelihood = -1368.6106  
Iteration 1:   log likelihood = -1320.8556  
Iteration 2:   log likelihood = -1320.4348  
Iteration 3:   log likelihood = -1320.4348  

Poisson regression                              Number of obs     =        690
                                                LR chi2(2)        =     512.39
                                                Prob > chi2       =     0.0000
Log likelihood = -1320.4348                     Pseudo R2         =     0.1625

------------------------------------------------------------------------------
     PostUOF |        IRR   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      PreUOF |   1.127117    .006889    19.58   0.000     1.113695      1.1407
       Group |   1.834683   .1226155     9.08   0.000     1.609436    2.091455
       _cons |   .7874286    .043496    -4.33   0.000     .7066306    .8774652
------------------------------------------------------------------------------

So, the experimental group's UOF rate in the post evaluation is 1.46 times larger than the control group, right? Is there an easier way to convey this information for a non-technical audience?
Also what does the Pre UOF variable tell us? Does it tell us that the incident ratio in the post UOF increase, as the number of Pre UOF increase??? Is this information useful in any way to convey? Or it is just a co-variate and should not be reported?

Poisson answered the question of whether there is a difference between group in post UOF controlling for pre UOF counts. But I also need to know if there is just a significant increase or decrease in UOF incidents between pre and post times for the groups. To answer this question, I conducted pair ttest and signrank for each group. Both show an significant drecrease in UOF from pe to post. My question is, is there a way to do this treating my variables as count data and not just comparing means? At the end I want to be able to say something in the lines of. 1. There was an decrease in UOF incidents from pre to post for both groups. and 2. the differences bewteen pre and post is greater for the control group (its only an example). It is ok if I use two different model/test to answer my questions.

      
by Group,sort: ttest PreUOF=PostUOF
by Group,sort: signrank PostUOF=PreUOF

thank you,
Marvin