Markdown / Markstat / Sublime Text

[Malte Paul]

Dear Forum,

I am getting into Literate Data Analysis and find that the Markstat package is of great value. (Thanks @German Rodriguez).
I would have two practical questions:

#1 : The auxiliary files that are created clutter the working directories a lot. Is it possible to save them in a separate folder from the .stmd file? This also applies to tables/graphs that are created and then inserted with (e.g.) .include{figure.html}.

#2 : I would like to use a better performing text editor to edit the .stmd files that supports syntax highlighting. I have tried using Sublime Text 3 with (either/both) the StataEditor and the Multimarkdown packages. There is an good post on it https://maxmasnick.com/2017/09/22/markstat/ However, this does not work with strict code blocks under Markstat. Ideally i would like to combine highlighting the syntax for both (e.g. for Stata in the code blocks). This is possible for other languages (e.g. combining HTML with Python). Is this possible? What are alternatives / or experiences from other users?

Thanks a lot in advance?

Best,
Malte

[Phil Bromiley]

You didn't get a quick answer. Getting an answer depends on someone active using the package you're discussing. Your questions seem to be more about Markstrat than Stata. Another forum might be more likely to get you help.

It is quite possible that none of the active folks use MarkStrat.

[German Rodriguez]

Thanks Malte Paul, glad to hear you have found markstat valuable.

#1. Short answer is that all files generated by markstat are in the same folder as the stmd script, a decision made in the interest of simplicity. In an attempt to reduce clutter, recent versions of markstat remove all intermediate files unless you use the keep option. We do keep the smcl log and all graphs to enable nodo, which avoids having to rerun Stata when you make small edits to the narrative. Keeping the graphs also allows linking rather than embedding them in web pages. As a workaround, you could create a temporary working folder, copy and run the script there, and then retrieve the final output file before removing the folder. You would need to make sure you use the bundle option when the output is html.

#2. I have been looking at alternative editors that include syntax highlighting for Markdown and Stata. I think an essential feature is the ability to select and run Stata code from the editor, which makes the Stata editor the simplest multi-platform solution at this time. In addition to the Sublime note you cite, Max Masnick has a nice Atom setup described at https://maxmasnick.com/kb/markstat/. I added an experimental feature in markstat 2.3 to recognize code fences that spell out stata as required by his setup. Getting all the pieces to work together, however, can be rather challenging, particularly on Windows.

I look forward to hearing other takes on both issues.

[Anders Alexandersson]

German Rodriguez

I look forward to hearing other takes on both issues.

To me, the first issue about log files is more important than syntax highlighting. Especially, I find it tedious to get separate log files for error checking. How can we use markstat to log one set of files for the reporting as usual, and one additional set of files for error checking? If it is easy to do, an example would be very helpful. Especially, how do I log separately a call to a do-file from a "master" stmd file?

[German Rodriguez]

Hi Anders Alexandersson.

how do I log separately a call to a do-file from a "master" stmd file?

The markstat command works with a single log file, but Stata can have up to five open log files as long as you name them. This opens up many possibilities. For example a do file could open and close its own log:

// the file checks.do
 log using checks, name(checks) replace
 // run your checks here
 log close checks

A markstat script can call this do file, and will include its output, because by default everything is echoed to all open log files. (In this scenario I would probably log with the nomsg option to remove the header and footer.) If you want to hide the checks, you could have the markstat script turn its own log off before calling the do file, and then on again:

# The Markstat Script

```s/
    quietly log off
    do checks
    quietly log on
```

Main analysis goes here

This way the checks will not appear in the dynamic document, but they will be run and logged in checks.smcl every time the document is generated. Is this what you had in mind?

[Anders Alexandersson]

German Rodriguez Thank you, yes it's exactly what I want. I wrongly had quietly do checks. It's great that markstat can handle this. It was not clear to me from the documentation.

[German Rodriguez]

To be honest I didn't know either, as I was thinking in terms of just one log file. I also tried quietly do checks, but then you get no logs downstream. Live and learn.

[Malte Paul]

Thank you @German Rodriguez for the detailed answer. With respect to both points.

#1 : Thanks for the helpful work-around. I was why it doesnt work to save figures/tables in a sub-folder and then include them from the sub-folder (e.g. by using .include{$workingpath/figures/figure.html})? Generally, my goal would be to keep the .stmd files in a separate folder than all the output. Is this possible somehow?

#2: I think it would, technically, be possible to include various code types into Sublime? Afaik, the feature exists for other combinations through packages such as multimarkdown (e.g. markdown +Java) ? So i guess its a matter to convice these programmers to implement this?

I dont really understand the second part of your answer. Would you mind to elaborate what would need to be done to make this setup work ?

Thanks a lot in advance,
Malte

[German Rodriguez]

Some further answers for @Malte Paul.

#1 Yes, you can save figures/tables in a subfolder and include them from there. For example to generate a graph in a subfolder use Stata's graph export subfolder/filename.png and to include the graph in the document use Markdown's . This also works for .include subfolder/filename.html, but your example appears to include a global macro and braces, which won't work.

#2 Many extensible code editors allow users to contribute syntax highlighting grammars. The hard part is to communicate with Stata to run selected code. Max Malnick describes his atom setup, which uses Kyle Barron's stata-exec to send commands to Stata. Please follow these links for their explanation of the steps needed to set things up. As

[Malte Paul]

Thank you very much @German Rodriguez.
I will try both your solutions.
Thanks again for providing and maintaing Markstat!

[JeongHoon Min]

#2 : I would like to use a better performing text editor to edit the .stmd files that supports syntax highlighting. I have tried using Sublime Text 3 with (either/both) the StataEditor and the Multimarkdown packages. There is an good post on it https://maxmasnick.com/2017/09/22/markstat/ However, this does not work with strict code blocks under Markstat. Ideally i would like to combine highlighting the syntax for both (e.g. for Stata in the code blocks). This is possible for other languages (e.g. combining HTML with Python). Is this possible? What are alternatives / or experiences from other users?

With the exact same need of Malte Paul, I forked PlainNotes package of Sublime Text to support Stata & R code. You can install my fork of PlainNotes from https://github.com/jh-min/PlainNotes. The key part is to edit .sublime-syntax files in whatever markdown packages of Sublime Text (This can be done by using PackageResourceViewer).

Below are examples using MarkdownEditing with ayu theme (left) and PlainNotes (right), respectively (sorry for non-English characters but you can grab how it works).
If opening fence is either ```s, ```{s} or ```stata, fenced code block will highlight Stata code using Monokai Pro color scheme and support any key bindings working in Stata source(do-files).
For example, if you are using StataEditor, alt+d will send current line to Stata (I recommend another customization in README document of my fork, though).