Mistake in command to calculate natural logarithm

[Leon Nguyen]

Hello Statalist,

I have a problem with command calculating natural logarithm in Stata that is so confusing:

gen sal=tn1/ld11
    gen lsal=ln(sal)
    gen lnsal=ln(tn1/ld11)
    br if lsal != lnsal

There are 4,942 of 83,752 observations appear after command "br". They looks identical but when I put them into stochastic frontier model, the results are totally different.

Could some one help me to explain why that happen? Thanks alot!

[Nick Cox]

Reproducible example, please.

[Leon Nguyen]

Reproducible example, please.

Thanks Nick Cox for reply. Please see this .xlsx file.

[Joro Kolev]

Sounds to me like a precision issue.

Replace each -gen- command you have by -gen double-, and see whether the issue is still there.

[Maarten Buis]

The way to give examples on this list is discussed in the FAQ (see black bar on top of this page for a link).

Most of us don't open files of the type you gave us from random persons on the internet (and neither should you). For example, we don't know if your system has been infected by some virus.

[Nick Cox]

Sorry, no. Please read https://www.statalist.org/forums/stata on why posting spreadsheet attachments is not as helpful as you hope. For a start,.many people here can't or won't read spreadsheet attachments.

What your data are like in an Excel spreadsheet has no bearing on an apparent mistake in Stata What we need is more like the results of

dataex tn1 ld11 sal lsal lnsal if lsal != lnsal

[Leon Nguyen]

Sounds to me like a precision issue.

Replace each -gen- command you have by -gen double-, and see whether the issue is still there.

Works nicely Joro Kolev. Thanks very much.
Is that any recommendation that for natural logarithm we should use -gen double- instead of-gen-?

[Carlo Lazzaro]

Leon:
the rule of thumb is to invoke -double- whenever precision is an a priori issue (or to fix problems like the on you complained about, as Joro helpfully recommended).

[Leon Nguyen]

The way to give examples on this list is discussed in the FAQ (see black bar on top of this page for a link).

Most of us don't open files of the type you gave us from random persons on the internet (and neither should you). For example, we don't know if your system has been infected by some virus.

Thanks Maarten Buis and Nick Cox. I am newbie here, and I will remember the way upload data next time

[Joro Kolev]

Works nicely Joro Kolev. Thanks very much.
Is that any recommendation that for natural logarithm we should use -gen double- instead of-gen-?

You use -double- when you are dealing with either very big, or very small numbers.

This is not science, but I would answer Yes to your questions, I always use double when I deal with logarithms and exponents (the inverse function of the logarithm).

In my own experience I have encountered similar problem when I calculate cumulative returns. For cumulative returns you need products , and Stata's -egen- has only sums, so one way to do it is to pass through logarithms. And then I noticed that I need to do the calculation in high precision (that is specify -double-) or otherwise the calculation produces nonsense.

[Leon Nguyen]

@Carlo: Good rule! I haven't noticed that issue until today and actually found that unintentionally. Should be more careful, I guess.

[Nick Cox]

I have never yet noticed a scientific conclusion hinging on whether a logarithmic transformation of a variable was stored as float or double.

[Joro Kolev]

I have never yet noticed a scientific conclusion hinging on whether a logarithmic transformation of a variable was stored as float or double.

Nick, the issue is more the precision that one is using in the intermediate calculations, and not the precision in which one saves the final result. My guess for the original example in #1 is that sal=tn1/ld11 is a very small number for some observations, hence the huge difference between the one shot calculation ln(sal=tn1/ld11) vs the two step calculation of firstly generating the ratio in low precision, and then in second step applying the log.

[Leon Nguyen]

I would like to post my data here as maybe someone want to take a closer look. The results as you can see is not really different, but Stata treat them differently. Especially in some sensitive model (in my case is Stochastic Frontier Analysis), one can help model converging and other one just mess up.

* Example generated by -dataex-. To install: ssc install dataex
clear
input double(tn1 ld11) float(sal lsal lnsal)
 22536   965  23.35337  3.150741  3.150741
 11051   125    88.408 4.4819627  4.481962
 34702   959  36.18561 3.5886614  3.588662
 84650   919  92.11099  4.522994 4.5229945
 73931   890  83.06854  4.419666 4.4196663
 92359  1080  85.51759  4.448722 4.4487224
 82501  1063  77.61148  4.351715 4.3517156
  7491   180  41.61666  3.728501 3.7285006
 21000   299 70.234116  4.251834 4.2518344
 16927   330  51.29394  3.937573 3.9375725
 13667   253  54.01976   3.98935   3.98935
 11682   220      53.1  3.972177  3.972177
 10580   228  46.40351  3.837375  3.837375
 47812   338 141.45563  4.951986  4.951986
 52096   372 140.04301  4.941949   4.94195
  3208   138 23.246376 3.1461494  3.146149
 12499   372  33.59946   3.51451   3.51451
  1906    44  43.31818 3.7685726  3.768572
 26563   330  80.49394 4.3881817  4.388182
 11326   235  48.19574  3.875271 3.8752706
 20481   295  69.42712  4.240278 4.2402773
 28578   290  98.54483 4.5905113  4.590512
  5181    42 123.35714  4.815084  4.815084
 28759   436 65.961006  4.189064 4.1890635
 23998   326 73.613495 4.2988286  4.298828
 10026   201   49.8806  3.909632  3.909632
 76398  1182  64.63452 4.1687484  4.168749
 30683   209 146.80861   4.98913   4.98913
 27359   225 121.59556    4.8007  4.800701
   510     7  72.85714  4.288501 4.2885003
 15886    39  407.3333  6.009632  6.009632
 20341    70 290.58572  5.671898  5.671899
 12356   272  45.42647  3.816095  3.816095
  7447    98   75.9898  4.330599 4.3305993
6793.4    69  98.45507 4.5896006    4.5896
  5631   116   48.5431  3.882452  3.882452
  5029   112  44.90179 3.8044775  3.804478
  1440    31  46.45161  3.838411  3.838411
   706    20      35.3  3.563883  3.563883
  2119    45  47.08889  3.852037  3.852037
 21494   280  76.76428 4.3407397 4.3407393
  8309   220  37.76818  3.631467  3.631467
  1589    46  34.54348  3.542219  3.542219
  2803    43  65.18604 4.1772456  4.177245
  8038   160   50.2375  3.916762 3.9167616
  3423    92  37.20652  3.616484  3.616484
   999    43  23.23256  3.145555 3.1455545
 11900   165 72.121216  4.278348 4.2783484
   508    21 24.190475  3.185959  3.185959
 59801  1468  40.73638 3.7071214 3.7071216
   540     7  77.14286  4.345659 4.3456593
  1392    30      46.4  3.837299 3.8372996
  6676    97 68.824745  4.231563 4.2315636
   982    20      49.1  3.893859  3.893859
  2178    40     54.45  3.997283  3.997283
  5513   166 33.210842 3.5028765  3.502876
  7593   102  74.44118  4.310009 4.3100095
  4236    49  86.44898  4.459554 4.4595547
  7144    86  83.06977 4.4196806  4.419681
   696    18  38.66667  3.654978  3.654978
 11719   265  44.22264  3.789237  3.789237
  1005    29 34.655174  3.545447  3.545447
 13532   289  46.82353  3.846386  3.846386
 41727   798  52.28947  3.956795  3.956795
   462     9  51.33333 3.9383404   3.93834
  1239    27  45.88889  3.826223  3.826223
  1110    25      44.4  3.793239 3.7932396
  9347   219  42.68037  3.753739  3.753739
   834    14  59.57143  4.087176 4.0871763
  1834    26  70.53846 4.2561584  4.256158
  7808   164  47.60976 3.8630376  3.863038
   790    51 15.490196  2.740207 2.7402074
   831    21  39.57143  3.678107 3.6781075
   580    30 19.333334 2.9618306  2.961831
  3522    89  39.57303  3.678148  3.678148
  2034    40     50.85   3.92888   3.92888
  4109    44  93.38636 4.5367455  4.536745
   248     7  35.42857  3.567519 3.5675185
  5378   145 37.089657  3.613338  3.613338
   813    18  45.16667  3.810359 3.8103595
   776    17  45.64706  3.820939  3.820939
   173     5      34.6  3.543854 3.5438535
   985    30 32.833332  3.491444  3.491444
 15413   287  53.70383  3.983484 3.9834845
 17827   297  60.02357 4.0947375  4.094737
  1074    37 29.027027  3.368227 3.3682275
  1031    20     51.55  3.942552  3.942552
 944.3    28    33.725 3.5182395  3.518239
  2492    70      35.6  3.572346 3.5723455
  3028    36  84.11111 4.4321384  4.432139
820559 23491 34.930782  3.553368 3.5533686
 11670    81 144.07408  4.970327  4.970328
  5602   126  44.46032  3.794597  3.794597
  1672    31  53.93548  3.987789 3.9877884
 10895   340 32.044117  3.467114 3.4671135
 41195   570  72.27193  4.280436 4.2804356
  2390    30  79.66666 4.3778515  4.377851
  2445    25      97.8 4.5829244  4.582925
  6095   150  40.63334  3.704589  3.704589
  4310    58  74.31035   4.30825 4.3082504
end