Can we determine the significance of changes over time using cross-sectional data?

[Monzur Alam]

Dear Statalisters,

I am using data from the 2011 and 2014 Demographic Health Surveys (these are repeated cross sections) to see the aggregate trends in intimate partner violence across type of residence (rural and urban). Does cross sectional data allow us to estimate the significance of the change? E.g., if the prevalence of violence in urban areas in 2011 was 13% and 10% in 2014, is there anyway we could carry out a statistical test in Stata to determine if this change is significant?

Thanks,

Monzur

[Carlo Lazzaro]

Monzur:
you can consider plugging -i.year-as a categorical variable among predictors of your regression.
As usual, more details about your dataset can help interested listers to reply (more) positively.

[Monzur Alam]

Monzur:
you can consider plugging -i.year-as a categorical variable among predictors of your regression.
As usual, more details about your dataset can help interested listers to reply (more) positively.

Thank you Carlo. I have posted an extract of the dataset to give listers an idea of how it is structured. As mentioned earlier it is a repeated cross-section. I have used the tab command to provide descriptive statistics of how attitudes justifying domestic violence have changed over time. But I would like to statistically test if this change is significant.

cluster_id	household_id	person_id	loc	justifies_domviol	year
9	24	2	rural	0	2011
433	55	1	rural	1	2011
336	21	2	rural	0	2011
326	109	4	urban	0	2011
540	113	2	urban	0	2011
212	26	1	rural	0	2014
393	25	2	rural	1	2014
86	62	7	rural	0	2014
137	144	1	urban	1	2014

.  ta justifies_domviol year, col nofreq

justifies_ |         year
   domviol |      2011       2014 |     Total
-----------+----------------------+----------
         0 |     68.01      68.70 |     68.50 
         1 |     31.99      31.30 |     31.50 
-----------+----------------------+----------
     Total |    100.00     100.00 |    100.00

Is this what you are suggesting?

. ologit justifies_domviol i.year

Iteration 0:   log likelihood = -37939.106  
Iteration 1:   log likelihood = -37937.732  
Iteration 2:   log likelihood = -37937.732  

Ordered logistic regression                     Number of obs     =     60,894
                                                LR chi2(1)        =       2.75
                                                Prob > chi2       =     0.0973
Log likelihood = -37937.732                     Pseudo R2         =     0.0000

-----------------------------------------------------------------------------------
justifies_domviol |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
------------------+----------------------------------------------------------------
             year |
            2014  |  -.0318009   .0191663    -1.66   0.097    -.0693663    .0057644
------------------+----------------------------------------------------------------
            /cut1 |   .7543742   .0161146                      .7227902    .7859582
-----------------------------------------------------------------------------------

[Carlo Lazzaro]

Monzur:
yes, but I expected other predictors to be included in the right-hand side of your regression equation.
Moreover, I fail to get why you used -ologit- instead of -logit-, as your regressand does not seem ordered.

[Monzur Alam]

Monzur:
yes, but I expected other predictors to be included in the right-hand side of your regression equation.
Moreover, I fail to get why you used -ologit- instead of -logit-, as your regressand does not seem ordered.

Carlo:
Thank you for pointing that out- it should be logit. Is there a valid statistical test for significance if I use only bivariate descriptive statistics (not multivariate regression) to present trends over time? This is for a very lay audience, who would be interested in change in attitudes over time (without going into multivariate analysis) and whether that change is significant.

[Carlo Lazzaro]

Monzur:
I think you should go -logit-, as descriptive statistics cannot give back statistical significance.

[Monzur Alam]

Monzur:
I think you should go -logit-, as descriptive statistics cannot give back statistical significance.

Carlo, thank you for your very helpful responses. I am still a little confused though. I came across a report where the researchers use a Z-test to test for differences between two proportions from two independent samples (from 2011 and 2015), and then interpreting this as the change over time being significant (or not). Would this be a valid interpretation? I have attached one page, in case it helps clarify their approach.

[Carlo Lazzaro]

Monzur:
if you do not have other predictors, you can go that way.
Obviously, this approach does not take into account the role of other predictors (made explicit or implicit) in explaining the reason of changes across time.

[Monzur Alam]

Thanks for patiently answering all my questions, Carlo! Yes, this is a rather crude account of changes across time.