"Convergence not achieved", are results valid?

[Ilse van Vliet]

Good day,

After teffects aipw Stata keeps iterating. However, after x amount of iterations I do get a table of results, including the warning "Convergence not achieved.". Are these results valid to present in a paper?

(I read on this forum that this is caused by running a too complicated model, but to me all variables seem necessary.)

[Andrew Musau]

After teffects aipw Stata keeps iterating. However, after x amount of iterations I do get a table of results, including the warning "Convergence not achieved.". Are these results valid to present in a paper?

No. You have to address the convergence issues one way or the other.

[Carlo Lazzaro]

Ilse:
as an aside to Andrew's excellent advice, you may want to add one predictor at a time and see when the convergence issue creeps up.

[Weiwen Ng]

The longer explanation: in maximum likelihood estimation, the software is trying to find the values of all the betas that maximize the likelihood (basically, trying to find the betas that maximize the probability that you got your data - I’m probably oversimplifying that, but it’s something along those lines).

the algorithm iterates until it converges. If it doesn’t converge, then it doesn’t have valid results. Some of the non convergence problems can be identified, but often they are very tricky to diagnose. You can try to report your results here and we can see if anything can be said, but unfortunately there’s no guarantee.

[Ilse van Vliet]

Andrew: thank you for the clear answer

Carlo: I tried to figure out which variable specifically causes the convergence issues, but it looks like it also depends on the combination of variables in de model.

Weiwen: Thank you in advance if you would take the time to a look at my data. (context below).

* Example generated by -dataex-. For more info, type help dataex
clear
input double nomem_encr float treatment double anxious byte wave float anxiousprev double(leeftijd geslacht herkomstgroep oplcat dutch cancer sted nettoink) float bmi double(woning work religious)
800009 0 0  9 . 61 1 0 4 2 . 4 2600  31.3 0 0 0
800009 0 0 10 0 62 1 0 4 2 0 4 2700  29.7 0 0 0
800009 0 0 11 0 63 1 0 4 2 0 4 2700  29.4 0 0 0
800009 0 0 12 0 64 1 0 4 2 0 4 2700    30 0 0 0
800009 0 0 13 0 65 1 0 4 2 0 4 2700  30.3 0 0 0
800009 0 0 14 0 66 1 0 4 2 0 4 3100    31 0 0 0
800015 0 0  3 . 47 1 2 5 2 0 0 2220  28.4 0 1 1
800015 0 2  4 0 48 1 2 5 2 0 0 2181    28 0 1 1
800015 0 1  5 2 49 1 2 5 2 0 0 2179  26.6 0 1 1
800015 0 4  6 1 50 1 2 5 2 0 0 2165    27 0 1 1
800015 0 0  7 4 51 1 2 5 2 0 0 2173  27.3 0 1 1
800015 0 0  9 . 54 1 2 5 2 0 0 2309  27.3 0 1 1
800015 0 0 10 0 55 1 2 5 2 0 0 2566    27 0 1 1
800015 0 1 11 0 56 1 2 5 2 0 0 2660  27.3 0 1 1
800015 0 0 12 1 57 1 2 5 2 0 0 2756    27 0 1 1
800015 0 0 13 0 58 1 2 5 2 0 0 2885  27.3 0 1 1
800015 0 0 14 0 59 1 2 5 2 0 0 2945    27 0 1 1
800018 0 1  6 . 27 1 0 3 . 0 2 1550  27.8 0 . .
800033 0 2  2 . 16 1 1 0 1 0 3    0  21.3 1 0 1
800033 0 1  4 . 18 1 1 2 1 0 3    0  20.6 1 1 1
800033 0 2  5 1 19 1 1 2 1 0 1    0  21.7 1 1 1
800033 0 1  6 2 20 1 1 2 . 0 1    0    22 1 . .
800042 0 2  1 . 32 2 0 2 2 0 2  180  26.6 0 1 1
800042 0 2  2 2 33 2 0 2 2 0 2  180  25.6 0 1 1
800042 0 1  3 2 34 2 0 2 2 0 2  180  25.6 0 1 1
800042 0 1  4 1 35 2 0 2 2 0 2  180  25.9 0 1 1
800042 0 1  5 1 36 2 0 2 2 0 2  180  25.9 0 1 1
800042 0 2  6 1 37 2 0 2 2 0 2  180  25.9 0 1 1
800042 0 2  7 2 38 2 0 2 2 0 2  180  25.9 0 1 1
800042 0 1  9 . 41 2 0 2 2 0 2  180  24.9 0 0 1
800045 0 0  1 . 65 1 . 1 2 0 3 2500  24.5 0 . 1
800054 0 0  9 . 70 2 0 1 2 0 3 1239  27.1 1 0 1
800054 0 0 10 0 71 2 0 1 2 0 3 1239  29.7 1 0 1
800054 0 0 11 0 72 2 0 1 . 0 3 1239  28.1 1 0 .
800054 0 0 12 0 73 2 0 1 2 0 3 1250  28.1 1 0 1
800057 0 0  1 . 32 1 0 5 2 0 0 1750  21.7 0 1 1
800057 0 0  2 0 33 1 0 5 2 . 0 1750  21.7 0 1 1
800057 0 0  3 0 34 1 0 5 2 . 0 1750  21.2 0 1 1
800057 0 0  4 0 35 1 0 5 2 . 0 2750  21.7 0 1 1
800057 0 0  5 0 36 1 0 5 2 0 0 2750  21.7 0 1 1
800057 0 0  6 0 37 1 0 5 2 0 0 2750    23 0 . 1
800057 0 0  7 0 38 1 0 5 2 0 0 2750    23 0 1 1
800057 0 0  9 . 41 1 0 5 2 1 0 2750  24.2 0 1 1
800057 0 0 10 0 42 1 0 5 2 1 0 2750  25.5 0 1 1
800057 0 0 11 0 43 1 0 5 2 0 0 2750  25.5 0 1 1
800057 0 0 12 0 44 1 0 5 2 0 0 4250  25.5 0 1 1
800057 0 0 13 0 45 1 0 5 2 0 0 4250  26.8 0 1 1
800058 0 2 13 . 23 2 0 2 2 0 0  250  25.5 1 1 1
800058 0 2 14 2 24 2 0 4 . 0 0  750  25.2 1 0 .
800085 0 2  9 . 39 1 0 3 2 0 4 1958  23.5 0 . 1
800085 0 2 10 2 40 1 0 3 2 0 4 2250  25.5 0 1 1
800085 0 3 12 . 42 1 0 3 2 0 4 2750  25.7 0 . 1
800100 0 2  9 . 26 2 2 0 2 0 0    0    27 1 1 0
800100 0 0 10 2 27 2 2 0 2 0 0    0  21.5 1 0 0
800100 0 4 11 0 28 2 2 3 2 0 0    0    27 1 0 0
800100 0 5 12 4 29 2 2 3 1 0 0    0  29.3 1 0 0
800100 0 0 13 5 30 2 2 3 2 0 0 1200  27.3 1 1 0
800100 0 1 14 0 31 2 2 3 . 0 0 1800  25.4 1 1 .
800115 0 0 10 . 49 1 2 3 2 0 1 1250 131.4 1 . 0
800119 0 0  1 . 57 2 1 1 1 1 2  250  29.4 1 0 0
800119 0 1  2 0 58 2 1 1 1 1 2  250    27 1 0 0
800119 0 0  3 1 59 2 1 1 1 0 2  250  29.4 1 0 0
800119 0 0  4 0 60 2 1 1 0 0 2  250  28.7 1 0 0
800119 0 0  5 0 61 2 1 1 1 0 2  250    28 1 0 0
800119 0 0  6 0 62 2 1 1 0 0 2  500  28.4 1 0 0
800119 0 1  7 0 63 2 1 1 0 0 2  500  29.8 1 0 0
800119 0 0  9 . 66 2 1 1 1 0 2  500  29.8 0 0 0
800119 0 0 12 . 69 2 1 1 1 0 2  500  31.1 0 0 0
800119 0 0 13 0 70 2 1 1 1 0 2  500  31.1 0 0 0
800119 0 2 14 0 71 2 1 1 1 0 2  500  31.1 1 0 0
800131 0 2  3 . 55 2 0 3 2 0 4    .  25.5 0 1 1
800131 0 4  4 2 56 2 0 3 2 0 4    .  27.6 0 1 1
800131 0 2  5 4 57 2 0 3 2 0 4    .  23.6 0 1 1
800131 0 2  6 2 58 2 0 3 2 0 4    .  24.7 0 1 1
800131 0 1  7 2 59 2 0 3 2 0 4    .  27.2 0 1 1
800131 0 1  9 . 62 2 0 3 2 0 4    .    29 0 0 1
800131 0 1 10 1 63 2 0 3 2 0 4    .  28.3 0 0 1
800131 0 2 11 1 64 2 0 3 2 0 4    .  26.1 0 0 1
800131 0 2 12 2 65 2 0 3 2 0 4    .  26.9 0 0 1
800131 0 2 13 2 66 2 0 3 2 0 4    .  22.5 0 0 1
800131 0 1 14 2 67 2 0 3 2 0 4    .  22.5 0 0 1
800158 0 0  1 . 38 1 0 3 2 0 3 1900  26.9 0 1 0
800158 0 2  2 0 39 1 0 3 . 0 3 1900  27.2 0 . .
800158 0 1  3 2 40 1 0 3 . 0 3 2000  27.8 0 . .
800158 0 1  4 1 41 1 0 3 2 0 3 2000  28.4 0 1 0
800158 0 1  5 1 42 1 0 3 2 0 3 2000    29 0 . 0
800161 0 1  6 . 42 1 0 4 2 0 2 2750  25.7 0 1 1
800161 0 0  7 1 43 1 0 4 2 0 2 2750  25.7 0 . 1
800161 0 0  9 . 46 1 0 4 2 0 2 3250    26 1 1 1
800161 0 0 10 0 47 1 0 4 2 0 2 3250  26.6 0 1 1
800161 0 0 11 0 48 1 0 4 2 0 3 3250  26.9 0 1 1
800161 0 0 12 0 49 1 0 4 2 0 3 3250  24.5 0 . 1
800161 0 0 13 0 50 1 0 4 2 0 3 4250  25.1 0 . 1
800161 0 0 14 0 51 1 0 4 2 0 3 5000    26 0 1 1
800170 0 0  1 . 49 2 0 3 2 0 3 1037  21.9 0 1 0
800170 0 0  2 0 50 2 0 3 2 0 3 1037  22.3 0 1 0
800170 0 0  3 0 51 2 0 3 2 0 3 1150  22.3 0 1 0
800170 0 2  4 0 52 2 0 3 2 0 3 1160  24.2 0 1 0
800170 0 0  5 2 53 2 0 3 2 0 3 1160  24.2 0 1 0
800170 0 0  6 0 54 2 0 3 2 0 3 1498  25.4 0 1 0
800170 0 0  7 0 55 2 0 3 2 0 3 1758  24.2 0 1 0
800170 0 0  9 . 58 2 0 3 2 0 3 1800  26.6 0 1 0
800170 0 0 10 0 59 2 0 3 2 0 3 1800  27.3 0 1 0
800170 0 0 11 0 60 2 0 3 2 0 1 1800  27.3 0 1 0
800170 0 0 12 0 61 2 0 3 2 0 1 2100  24.2 0 1 0
800170 0 0 13 0 62 2 0 3 2 0 1 2300  25.8 0 1 0
800170 0 0 14 0 63 2 0 3 2 0 1 2300  25.8 0 1 1
800186 0 2  2 . 69 2 0 1 2 0 3  750  24.5 0 0 0
800186 0 2  3 2 70 2 0 1 2 0 3  750  24.5 0 0 0
800186 0 2  4 2 71 2 0 1 2 0 3  750  24.5 0 0 0
800186 0 1  5 2 72 2 0 1 2 0 3  750  24.2 0 0 0
800186 0 3  6 1 73 2 0 1 1 0 3 1250  24.6 0 0 0
800186 0 2  7 3 74 2 0 1 1 0 3 1600  24.2 0 0 0
800186 0 2  9 . 77 2 0 1 1 0 3 1600  24.2 0 0 0
800186 0 2 10 2 78 2 0 1 1 0 3 1600  24.2 0 0 0
800186 0 3 11 2 79 2 0 1 1 0 2 1600  24.2 0 0 0
800186 0 3 12 3 80 2 0 1 1 0 2 1600  24.2 0 0 0
800186 0 2 13 3 81 2 0 1 1 0 2 1750  24.2 0 0 0
800186 0 2 14 2 82 2 0 1 1 0 2 1750  24.2 0 0 0
800201 0 2  1 . 34 1 0 0 2 0 1  900  22.7 1 1 1
800201 0 4  2 2 35 1 0 0 2 . 1 1500  24.7 1 1 1
800201 0 3  3 4 36 1 0 0 2 . 1 1000  26.4 1 1 1
800201 0 2  4 3 37 1 0 0 2 . 1 1000  23.4 1 1 1
800201 0 2  5 2 38 1 0 0 2 . 1 1000  22.1 1 1 1
800201 0 2  6 2 39 1 0 0 2 0 1 1000  25.1 1 1 1
800201 0 3  7 2 40 1 0 0 2 0 1 1000  26.1 1 1 1
800201 0 2  9 . 43 1 0 0 2 0 1 1250  23.7 1 1 1
800201 0 2 10 2 44 1 0 0 2 0 1 1700    20 1 1 1
800201 0 0 11 2 45 1 0 0 2 0 3 1700  20.4 1 1 1
800201 0 2 12 0 46 1 0 3 2 0 3 1900  25.1 1 1 1
800201 0 2 13 2 47 1 0 3 2 0 3 1900  26.7 1 1 1
800201 0 2 14 2 48 1 0 3 2 0 3 1900  27.4 1 1 1
end

Context: I am investigating the effect of non-clinical hallucinogens use (treatment) on anxiety using a longitudinal dataset with about 10K individuals. Treatment remains 1 in the years after an individual used hallucinogens. I consider anxiety level in the previous year (anxiousprev) as a confounder that should definitely be included in the model. However, doing so among others creates convergence issues.

Before I came up with the idea to include anxiousprev, I started with this command (which does not lead to convergence issues):

teffects aipw (anxiety cancer geslacht herkomstgroep c.leeftijd dutch) (treatment c.leeftijd cancer geslacht herkomstgroep sted, probit), wnls vce(cluster nomem_encr) osample(test2)
drop if test2==1
teffects aipw (anxiety cancer geslacht herkomstgroep c.leeftijd dutch) (treatment c.leeftijd cancer geslacht herkomstgroep sted, probit), wnls vce(cluster nomem_encr) aequations

I also ran other combinations of variables in the outcome and treatment models and the ATE remained between -0.033 and -0.037.
But then I started thinking about adding anxiousprev and convergence issues started. After some investigation, my finding is that herkomstgroep, sted, woning, oplcat, c.nettoink, work are causing convergence issues when included in the treatment model. And c.leeftijd, c.bmi, dutch and anxiousprev##cancer are causing issues when included in the outcome model. When anxiousprev is only included in the OM it also causes issues (when included in both no problem occurred)

Not all these variables should per definition be included in my model. Especially defining the right TM is hard when almost every variable causes convergence issues. Based on earlier research on relationships between these variables, this would be my "perfect" model:

teffects aipw (anxious anxiousprev geslacht i.herkomstgroep c.leeftijd oplcat cancer c.bmi religious work) (treatment anxiousprev c.leeftijd geslacht herkomstgroep oplcat cancer sted, probit), wnls vce(cluster nomem_encr)

[Andrew Musau]

Does the dataex include all observations in your dataset? I ask because there is no need to diagnose convergence problems with a subset of the data. If it does not, upload a .dta file with all the data. Then it may be possible to try out some things, but as Weiwen Ng points out in #4, there are no guarantees of success.

[Weiwen Ng]

Or show the results where the model didn't converge. Generally, problematic variables have missing standard errors. The thing is, what you should do about them aside from potentially omitting the offending independent variable can be a bit less clear. People with better knowledge of PSM than I have might be able to say something.

[Ilse van Vliet]

Andrew Musau -dataex- did not include all my observations (there are 73249), my apologies. I tried to attach my data file but receive an error when uploading.

Weiwen Ng Most of the times all results were shown, like in this example:

teffects aipw (anxious geslacht c.leeftijd cancer anxiousprev) (treatment c.leeftijd geslacht anxiousprev, probit), wnls vce(cluster nomem_encr) osample(test2)
drop if test2==1
teffects aipw (anxious geslacht c.leeftijd cancer anxiousprev) (treatment c.leeftijd geslacht anxiousprev, probit), wnls vce(cluster nomem_encr) aequations

Output:

Treatment-effects estimation                    Number of obs     =     49,946
Estimator      : augmented IPW
Outcome model  : linear by WNLS
Treatment model: probit
                        (Std. err. adjusted for 10,940 clusters in nomem_encr)
------------------------------------------------------------------------------
             |               Robust
     anxious | Coefficient  std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
ATE          |
   treatment |
   (1 vs 0)  |  -.1123522   .0425311    -2.64   0.008    -.1957117   -.0289927
-------------+----------------------------------------------------------------
POmean       |
   treatment |
          0  |   1.106748   .0090771   121.93   0.000     1.088957    1.124538
-------------+----------------------------------------------------------------
OME0         |
    geslacht |   .1784377   .0220688     8.09   0.000     .1351836    .2216917
    leeftijd |  -.0064248   .0008532    -7.53   0.000     -.008097   -.0047526
      cancer |     .22329   .0784614     2.85   0.004     .0695084    .3770715
 anxiousprev |   .4393926   .0151831    28.94   0.000     .4096343    .4691508
       _cons |   .6734251   .0462778    14.55   0.000     .5827223    .7641279
-------------+----------------------------------------------------------------
OME1         |
    geslacht |   -.166634   .0993075    -1.68   0.093    -.3612731    .0280052
    leeftijd |  -.0341165   .0097277    -3.51   0.000    -.0531824   -.0150505
      cancer |   -1.05614   .3723185    -2.84   0.005    -1.785871   -.3264096
 anxiousprev |   .8542903   .1132259     7.55   0.000     .6323718    1.076209
       _cons |   2.257308   .6379213     3.54   0.000     1.007005    3.507611
-------------+----------------------------------------------------------------
TME1         |
    leeftijd |  -.0291052   .0033331    -8.73   0.000    -.0356379   -.0225724
    geslacht |  -.4137726   .1097534    -3.77   0.000    -.6288853   -.1986598
 anxiousprev |   .0998562   .0279967     3.57   0.000     .0449838    .1547287
       _cons |  -.8884305   .2102523    -4.23   0.000    -1.300517   -.4763436
------------------------------------------------------------------------------
Warning: Convergence not achieved.