Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Using file command to link Stata with LaTeX

    Hello everyone.

    Basic stuff: I'm using a Mac, Stata/MP 13.1 version, and this is my first post.

    Well, the basic idea is that I want to link Stata outputs automatically to LaTeX (for my dissertation). My approach is to write a LaTeX-friendly log file, in which I could specify global macros that LaTeX could find anywhere in the document I'm writing.

    For example,

    Code:
    sysuse auto, clear
    mean price
    matrix R = r(table)
    
    * I will write a LaTeX-friendly log file with the result previously found
    quietly log using outputstata.txt
    di in w "\def\price#1{\gdef\@price{#1}}"     // LaTeX code for defining a command
    di in w `"\price{`=string(`=R[1,1]',"%20.0fc")'}"'  // LaTeX code for declaring the values of the command
    quietly log close
    This will end up with a log file that I can later use it as an \input{outputstata.txt} in LaTeX, and therefore, write a document with the global \@price (commas included), instead of writing the number on my own and coming back and forth to Stata and check if my answer has changed given my new procedures.

    Until here, I found that there is no problem at all and I have successfully compile this log file in LaTeX. Hooray!

    My problem came when I wanted to check if that global has been previously defined, so I could replace it with the new value. That is, I can't just use the before procedure to write an ado-file such as:

    Code:
    outputlatex, name(price) value(`=R[1,1]')
    With macros `name' and `value' because, as it is, this will only add and add and add new lines into my log file, even if I have already defined that same LaTeX global before.

    So, what I found is that there is a file command, with which you can read and write to and from any file.

    Therefore, my approach was the following,

    Code:
    program define outputlatex
        syntax , NAME(string) VALue(string) [NEW]
    
        * Open *
        if "`new'" == "new" {
             quietly log using "$results/outputstata.txt", name(latex) replace text
        }
        else {
             quietly log using "$results/outputstata.txt", name(latex) append text
        }
        
        * Check if there is a previous global *
        tempname myfile
        file open `myfile' using "$results/outputstata.txt", read write
        file read `myfile' line
        while r(eof) == 0 {
            if `"`line'"' == "\def\\`name'#1{\gdef\@`name'{#1}}" {
                file write `myfile' `"\\`name'{`=string(`value',"%20.0fc")'}"' _n
                * Close *
                file close `myfile'
                quietly log close latex
                exit
            }
            file read `myfile' line
        }
        file close `myfile'
    
        * If nothing was found *
        di in w "\def\\`name'#1{\gdef\@`name'{#1}}"
        di in w `"\\`name'{`=string(`value',"%20.0fc")'}"'
        * Close *
        quietly log close latex
    end
    What I do first is to check if I want a brand new log file. If I don't, then I will open the previously used log file, named here outputstata.txt. Then, I will check if I have already written a LaTeX global under the name `name'. I compare the complete line, to avoid the use of substr() or other string functions. If I do find a previous global, I will like to replace the value of `name' with the new `value'. I assume that if I found the first line, the second one will be the one I'm looking for replacement.

    AND HERE IS THE PROBLEM, this procedure does not replace the line completely. It stops at the first comma. How should I tell Stata that I want to whole line to be replaced? And more over, I formerly tried to replace the first line too, but I encountered that I could not write on the exact line in which I though I was, but the next one.

    So, any ideas of how can I code this ado-file with successful results? Maybe I'm not thinking it correctly from the first place, I there could be an easier more efficient and simple procedure to do. But that is what I come up with. So, any help is appreciated.

    Thanks a lot!
    Last edited by Ricardo Cantu; 24 Dec 2015, 03:21.

  • #2
    Unless you're planning to output all of the text directly from Stata as well, why not use some of the other available options that are already developed? For example, you could use StatWeave (http://homepage.stat.uiowa.edu/~rlenth/StatWeave/) to write everything in LaTeX and use the Java executable to run the code for you and compile the results into a single LaTeX document, texdoc
    Code:
    net describe texdoc, from(http://fmwww.bc.edu/RePEc/bocode/t)
    to do something similar directly from Stata, any one of several different user written programs to create LaTeX formatted output (e.g., estout, outreg2, latab, etc... all available from the SSC archives)? Given the last bit if what you mention, it sounds like StatWeave may be a good option since there are options available to process/supress the processing of the code chunks.

    Comment


    • #3
      Thanks a lot, Wbuchanan!

      I have read about texdoc and StatWeave and I'm sure I will find them very useful from now on!

      Although, I will still like now to know how to use the file write command. How do I make file write command do what I want it to do? I have read the help file and still I have some questions about how to use it efficiently and, more importantly, correctly.

      For example, how to make erase the line of reference completely? How can I replace it with the new values? Can you help me with that?

      The thing is that, given my LaTeX document and the current structure of my Stata ado-files, texdoc will not help me here.

      What I want is ONE log file with all my updated Stata results. So I can use them, in any way I want (not previously define as in texdoc) in my LaTeX document.

      Thanks!

      Comment


      • #4
        Ricardo Cantu I'm still not completely clear where the difficulty is. Perhaps you could run a small sample of your program, but use the commands:

        Code:
        set tracedepth 2
        set trace on
        Prior to it so we can see how the macros are being expanded. If your execution plan is to always overwrite previously existing results, why not use a pattern where you initalize a master log file in a master.do file. The master .do file then calls several slave.do files which contain isolated chunks of your analysis and could also create their own individual log files (in case you were just wanting to a review a single file). Then just use the display command to print the LaTeX global definitions into the log files. You may also want to look at the sjlatex package:

        Code:
        net desc sjlatex, from("http://www.stata-journal.com/production")
        It contains all of the LaTeX used for the StataJournal and StataPress publications, and there are already some commands available that aide in creating log files that are used to insert output into larger LaTeX documents. When I've used the file command for work with LaTeX, I've typically embedded a LaTeX template in an ADO and the program runs in order to generate LaTeX files that have specific formatting requirements with the appropriate values.

        The only other option I could think of at the moment is to check whether or not the filefilter command meets your needs. It allows you to search a file's contents and replace matching content with new content. It does, however, require the creation of a separate file (you are required to specify an input and output file). For a single search/replace this isn't a problem, but if you need to search/replace values multiple times in the same file you may want to specify tempfiles to be used to protect the original source/content from being overwritten accidentally.

        Comment


        • #5
          Ricardo,

          Seems to me that the difficulties are with the concurrent read-write operation,
          where the -file, read write- probably does not match your expectations.

          Instead of the read-write, you may read from one file and write to another file.
          Code:
          clear all
          capt log close all
          
          qui {
              
              log using outputstata.txt , replace text
              
              noi di "\def\latexcmdA#1{\gdef\@latexcmdA{#1}}"
              noi di "\latexcmdA{1}"
              noi di "\def\latexcmdB#1{\gdef\@latexcmdB{#1}}"
              noi di "\latexcmdB{2}"
              noi di "\def\latexcmdC#1{\gdef\@latexcmdC{#1}}"
              noi di "\latexcmdC{3}"
              
              log close
          }
           
          tempname myfile
          tempname myout
          
          capt file close `myfile'
          capt file close `myout'
          
          local name "latexcmdB"
          local value = c(pi)
          
          file open `myfile' using "outputstata.txt", read text    
          file open `myout'  using "outputstata.out", write text replace
          
          file read `myfile' line
          
          while ( r(eof) == 0 ) {
                  
              if regexm( `"`line'"', `"\\`name'{([0-9.]+)}"' ) {
              
                  file write `myout'  `"\\`name'{`value'}"' _n
              }
              
              else {
              
                  file write `myout' `"`line'"' _n
              }
              
              file read `myfile' line
          }
              
          file close `myfile'
          file close `myout'
          The files before and after replacement of argument to Latex command latexcmdB:
          Code:
          . type outputstata.txt
          \def\latexcmdA#1{\gdef\@latexcmdA{#1}}
          \latexcmdA{1}
          \def\latexcmdB#1{\gdef\@latexcmdB{#1}}
          \latexcmdB{2}
          \def\latexcmdC#1{\gdef\@latexcmdC{#1}}
          \latexcmdC{3}
          
          . type outputstata.out
          \def\latexcmdA#1{\gdef\@latexcmdA{#1}}
          \latexcmdA{1}
          \def\latexcmdB#1{\gdef\@latexcmdB{#1}}
          \latexcmdB{3.141592653589793}
          \def\latexcmdC#1{\gdef\@latexcmdC{#1}}
          \latexcmdC{3}

          Comment


          • #6
            Ricardo, since this thread is not closed, I add two comments.

            1) In your program definition in post #1 you open the same file at the same time by both -log, append- and -file, read write-.
            This seems unnecessary. You could instead first use -file- to do the replacement if the name argument exist, then if the name argument does not exist display new name using -log, append-. But, it might be better to avoid -file, read write-, because:

            2) According to -help file- "Rewriting a line of an text file works as expected only if the new and old lines are of the same length."

            This might give unexpected results and break you (latex) code. Let see what happens when the new argument is longer than the original:
            Code:
            qui {
                
                log using outputstata.txt , replace text
                
                noi di "\def\latexcmdB#1{\gdef\@latexcmdB{#1}}"
                noi di "\latexcmdB{2}"
                noi di "\def\latexcmdC#1{\gdef\@latexcmdC{#1}}"
                noi di "\latexcmdC{3}"
                
                log close
            }
             
            tempname myfile
            
            local name "latexcmdB"
            
            foreach value of numlist 9 20 300 4000 `=c(pi)' {
            
                file open `myfile' using "outputstata.txt", read write text    
            
                file read `myfile' line
            
                while ( r(eof) == 0 ) {
                        
                    if regexm( `"`line'"', `"\\`name'{#1}"' ) {
                    
                        file write `myfile'  `"\\`name'{`value'}"'_n
                    }
                    
                    file read `myfile' line
                }
                    
                file close `myfile'
            
                di _n _dup(5) "-" " `value' " _dup(5) "-" _n    
                type outputstata.txt
            }
            Then;
            Code:
            ----- 9 -----
            
            \def\latexcmdB#1{\gdef\@latexcmdB{#1}}
            \latexcmdB{9}
            \def\latexcmdC#1{\gdef\@latexcmdC{#1}}
            \latexcmdC{3}
            
            ----- 20 -----
            
            \def\latexcmdB#1{\gdef\@latexcmdB{#1}}
            \latexcmdB{20}
            def\latexcmdC#1{\gdef\@latexcmdC{#1}}
            \latexcmdC{3}
            
            ----- 300 -----
            
            \def\latexcmdB#1{\gdef\@latexcmdB{#1}}
            \latexcmdB{300}
            ef\latexcmdC#1{\gdef\@latexcmdC{#1}}
            \latexcmdC{3}
            
            ----- 4000 -----
            
            \def\latexcmdB#1{\gdef\@latexcmdB{#1}}
            \latexcmdB{4000}
            f\latexcmdC#1{\gdef\@latexcmdC{#1}}
            \latexcmdC{3}
            
            ----- 3.14159265359 -----
            
            \def\latexcmdB#1{\gdef\@latexcmdB{#1}}
            \latexcmdB{3.14159265359}
            dC#1{\gdef\@latexcmdC{#1}}
            \latexcmdC{3}
            If you still want to test the concurrent read-write approach, the following seems to work:
            Code:
            capt prog drop testIT
            prog def testIT
            
            syntax , NAME(string) VALue(string)
            
            tempname myfile
            
            local match = 0
            
            file open `myfile' using "outputstata.txt", read write text    
            
            file read `myfile' line
            
            while ( r(eof) == 0 ) {
                    
                if regexm( `"`line'"', `"\\`name'{#1}"' ) {
                
                    file write `myfile'  `"\\`name'{`value'}"'_n
                    local match = 1
                    continue , break
                }
                
                file read `myfile' line
            }
                
            file close `myfile'
            
            qui if ( `match' == 0 ) {
                
                log using "outputstata.txt", name(latex) append text
            
                noi display  `"\def\\`name'#1{\gdef\@`name'{#1}}"'
                noi display `"\\`name'{`=string(`value',"%20.0fc")'}"'
            
                log close latex
            }
            
            end
            Last edited by Bjarte Aagnes; 30 Dec 2015, 10:28.

            Comment


            • #7
              You did it, Bjarte Aagnes! Thanks a lot for your help!

              Let me just rewrite what my ado-file program looks like now.

              Few notes first:
              1. It was unnecessary my first -log, append-, as you said.
              2. I don't quite understand what -regexm- actually does. So, I changed your -if- condition with something that seems more logical to me.

              Code:
              program define outputlatex
              
                  syntax , NAME(string) VALue(string)
              
                  * Check existence of a previous value *
                  tempname myfile myout
                  file open `myfile' using "outputstata.txt", read write text
                  file read `myfile' line
                  while r(eof) == 0 {
                      if `"`line'"' == "\def\\`name'#1{\gdef\@`name'{#1}}" {
                          file write `myfile' `"\\`name'{`=string(`value',"%20.0fc")'}"' _n
                          noisily di in w `"\\`name'{`=string(`value',"%20.0fc")'}"'
                          local match = 1
                          continue, break
                      }
                      file read `myfile' line
                  }
                  file close `myfile'
                  
                  * Add value *
                  if "`match'" == "" {
                      quietly log using "outputstata.txt", name(latex) append text
              
                      di in w "\def\\`name'#1{\gdef\@`name'{#1}}"
                      di in w `"\\`name'{`=string(`value',"%20.0fc")'}"'
                      
                      log close latex
                  }
              
              end
              What I still don't understand is, if I'm at line `line' (according to my -if `"`line'"' == "\def\\`name'#1{\gdef\@`name'{#1}}"-), why when I use --file write-- it rewrites the next line and not the one --I believe-- I am at. That is, I need to find the previous line of that that I actually want to rewrite. But it works fine for my purposes now and I appreciate your help!

              Also, thanks wbuchanan for your insights and the sources you shared to me!

              Comment


              • #8
                Ricardo, regarding your last question in post #7:

                why when I use --file write-- it rewrites the next line and not the one --I believe-- I am at.
                My interpretation is:

                When using -file, text read write-, the write will be done from the current position, which will be at the end of the current line, including end-of-line character(s).

                To replace the current line, you must move back to the end of the previous line using -file seek-.

                Your current code could be changed to do this and in this case, where we don’t know the original value to match, the regex match may be more useful so I keep a slightly modified regex match.

                (Remember ref. post #6. According to -help file- "Rewriting a line of an text file works as expected only if the new and old lines are of the same length.")

                Code:
                file open `myfile' using "outputstata.txt", read write text
                file read `myfile' line
                
                local EOL = cond( c(os)=="Windows" , 2 , 1)
                
                while ( r(eof) == 0 ) {
                    
                    if regexm( `"`line'"', `"^[\]`name'{([0-9.]+)}$"' ) {
                        
                        file seek `myfile' query // leave scalar r(loc)
                        file seek `myfile' `= r(loc)-(length("`line'")+`EOL') '
                
                        file write `myfile'  `"\\`name'{`value'}"'_n
                        local match = 1
                        continue , break
                    }
                
                    file read `myfile' line
                }
                
                file close `myfile'
                Last edited by Bjarte Aagnes; 10 Jan 2016, 13:30.

                Comment

                Working...
                X