t-test confusion in significant meaning and conceptual framework

[Phuc Nguyen]

When reading a paper of Fan, 2021, I found two things that I feel it is strange

They conducted a t-test and they argue the results as below

Simply speaking, Econ, Soci, Ecol, and Util are four dependent variables. They noted :

"Except for resource utilization, the treated group have undergone pronounced changes"

resource utilization is Util in the Table

So I have two questions here:
(1) Why they noted like that? I saw all variables changed statistically significant after event

(2) The t-test is used to assess the statistical significance of the difference in means between the treatment and control groups, and to determine whether this difference is likely due to chance or to the intervention. A statistically significant difference indicates that the intervention had a significant impact on the outcome variable, while a non-significant difference suggests that the intervention did not have a significant impact.

So, why the author dont compare the difference of means of treated group and control group before and after event but they did compared control group with control group before and after and compare treated with treated groups before and after event day.

[Leonardo Guizzetti]

I can't access the article but I'll give my $0.02.

As I read the table, I am inclined to believe that they conducted a bunch of t-tests comparing pre- and post-intervention. As you note, all of these indicate some statistically significant change after intervention. In contrast to the authors, Util and Ecol both show declines after intervention, rather than just Util.

The table is not useful in trying to replicate the t-statistic calculations, which is unfortunate. I'm not an economist or econometrician, but I would think that a more meaningful comparison is to compare treated and control groups post-intervention and adjusting for the pre-intervention level.

[Carlo Lazzaro]

Phuc:
according to what the article reports, it seems that the (weird) ttest table is an intermediate step towards DID analysis.
It might be that reviewers asked authors to provide it in the revised version of the paper.

[Phuc Nguyen]

Phuc:
according to what the article reports, it seems that the (weird) ttest table is an intermediate step towards DID analysis.
It might be that reviewers asked authors to provide it in the revised version of the paper.

Carlo Lazzaro I have feeling it is the reason, unfortunately. It seems not to be a proper guidance to replicate that one from my point of view, make little sense to me...

[Carlo Lazzaro]

Phuc:
these kind of pre/post (or "mirror") studies were frequent fome some diseases 20 years ago.
Investgators measured a given outcome (or a set of outcomes) on the same patient before and after, say, a given therapy.
This way, being patient's other variables constant, each patient acted as the control for her/himself.
It's a while that I do not come across this kind of study, probably because nowadays regression outperforms this approach.
That said, if you disagree with the approach followed by Authors (who might have been forced to present Table 4), you can safely skip it, explaining why you do not agree with it (personally, instead of a series of ttest I woiuld have considered a regression).

[Phuc Nguyen]

Phuc:
these kind of pre/post (or "mirror") studies were frequent fome some diseases 20 years ago.
Investgators measured a given outcome (or a set of outcomes) on the same patient before and after, say, a given therapy.
This way, being patient's other variables constant, each patient acted as the control for her/himself.
It's a while that I do not come across this kind of study, probably because nowadays regression outperforms this approach.
That said, if you disagree with the approach followed by Authors (who might have been forced to present Table 4), you can safely skip it, explaining why you do not agree with it (personally, instead of a series of ttest I woiuld have considered a regression).

Thank you for the discussion, Carlo Lazzaro. I am trying to satisfy all the assumption of PSM-DiD, maybe I passed that one