Using external editors (e.g. UltraEdit) with Unicode support for Stata 14

[Dirk Enzmann]

[this is a continuation of the topic "Translate Stata-14 (utf8) do-file into Stata-13 (or earlier) do-file"]

Thanks to Friedrich Huebler new versions of rundo and rundoline scripts are available (see http://huebler.blogspot.ca/2008/04/stata.html) that allow to integrate external text editors with Stata.

However, in my environment (Windows 7, UltraEdit 22.10) they still do not work properly: Although it is no problem to properly edit do-files translated by unicode translate into utf8 coding in my external editor UltraEdit, when sending lines of code (using the rundolines program) or the do-file (using the rundo program) to Stata 14, umlauts (such as ü, ö, ä, ß) will not appear in utf8-coding in Stata. Instead, code such as

/* Using umlauts ö, ä, ü, ß: */

will appear as

/* Using umlauts �, �, �, �: */

in Stata.

[Friedrich Huebler]

The problem could be the encoding settings of your editor. I am not familiar with UltraEdit but the rundo and rundolines programs work fine with Notepad++, although it may be necessary to change the encoding. As an example, create a do-file with the following command, followed by a carriage return (if the command is not followed by a carriage return it is not executed by rundo).

di "ä, ö, ü, ß"

Example 1: Execute command with rundolines from a file encoded in ANSI.

. di "ä, ö, ü, ß"
ä, ö, ü, ß

Example 2: Execute command with rundolines from a file encoded in UTF-8 (with BOM).

. di "ä, ö, ü, ß"
ä, ö, ü, ß

Example 3: Execute command with rundolines from a file encoded in UTF-8 without BOM.

. di "ä, ö, ü, ß"
ä, ö, ü, ß

Example 4: Execute command with rundo from a file encoded in ANSI. The characters are not read correctly.

. di "�, �, �, �"
�, �, �, �

Example 5: Execute command with rundo from a file encoded in UTF-8 (with BOM). Stata does not recognize the command because it is preceded by an unknown character in the do-file.

. . di "ä, ö, ü, ß"
. di is not a valid command name
r(199);

Example 6: Execute command with rundo from a file encoded in UTF-8 without BOM.

. di "ä, ö, ü, ß"
ä, ö, ü, ß

Conclusion: With Notepad++ rundolines always works and rundo works if the encoding is set to "UTF-8 without BOM". I have tested the programs with 32-bit Windows 7 and 64-bit Windows 8.1.

[Dirk Enzmann]

Yes, the problem I encountered is (partly) due to the encoding settings of my editor (UltraEdit Version 22.10 under Windows 7). If you open an utf8-encoded do-file in UltraEdit, everything is fine and the rundo program (version 5.0) works fine. In case you start editing a new do-file you can save it as "UTF-8 without BOM". If you open a file that is not utf8-encoded you can convert it within UltraEdit (most likely "ASCII to UTF8 (unicode editing)" will work).

However, whether you are editing an utf8-encoded file or not, rundolines (version 5.0) will not copy utf8-encoded strings into the clipboard and Stata 14 will not read the characters correctly. I found a fix to rundolines that solved my problem although in some instances I encountered a problem I still can't solve (see below). The fix is as follows:

• Take care that UltraEdit's keyboard assignment to convert ASCII (or the file format that is standard for your UltraEdit version) to UTF8 is <Alt-u> (see UltraEdit's options to change the keyboard assignment).
• Take care that UltraEdit's keyboard assignment to close a file is <Alt-s>.
• Insert the following command lines at line 60 of the rundolines.au3 script (version 5.0) and compile the script using AutoIt3:

; ---------- fix for UltraEdit 22.10: ---------
; Open temporary file in editor
$tempfile3 = " " & $tempfile
Send("^o")
Sleep($clippause)
Send($tempfile3 & "{Enter}")
Sleep($clippause)
; Convert file to utf8 coding (assumes UltraEdit's keyboard assignment to convert ascii to utf8 is <Alt-u>)
Send("!u")
Sleep($clippause)
; Save and close temporary file (assumes UtraEdit's keyboard assignment to close a file is <Alt-s>)
Send("^s")
Send("!s")
; ---------- end fix for UltraEdit. ------------

The problem with this fix is that sometimes the script command

Send($tempfile3 & "{Enter}")

that should insert the file name into the "open file" dialogue window results in a file name with the first character stripped. I can't reproduce this problem and tried to fix it inserting some

Sleep($clippause)

commands. I would be great if someone knows why this occurs and can come up with a better fix.