MarkDoc: Tables out of matrices with the tbl command

[Sebastian Pehle]

Dear statalist,

iam interested in the "MarkDoc" package for quite a while now and looking for a way to obtain nicer tables than the ones generated by stata with the tab2 command and its output in the smcl log.
So i thought about converting the frequencies in a matrix and print the values to generate a tbl-table.
This is what ive accomplished so far:

global odir="youroutputdir"
****begin markdoc log************************************************************
cap erase $odir/markdoc.pdf
cap erase $odir/markdoc.md
cap erase $odir/markdoc.smcl
cap erase $odir/marktable.do
cap log close _all
quietly log using $odir/markdoc.smcl, replace smcl name(markdocN)

* We dont want the following in the log, so we turn it off
//OFF

mat matrix=(1,2\3,4)
mat list matrix
mat rownames matrix="row1" "row2"
mat colnames matrix="col1" "col2"

local cols=colsof(matrix)
local rows=rowsof(matrix)

local ++cols
local ++rows

quietly {
file open mktbl using $odir/marktable.do, write replace
    forvalues row=1/`rows' {         // every row of matrix

        forvalues col=1/`cols' {     // every column of matrix

            local colmax=`cols'-1     // local for last col ->put \ in there<-        
        
            if `row'==1 {              // put rownames in the first row

                if `col'==1 {
                    file write mktbl "tbl ( r ,  /// " _n // write tbl intro
                }
                
                if `col'<=`colmax' { // write info with comma
                    file write mktbl "`: word `col' of `: colnames matrix'', "
                }

                if `col'==`cols' { // write info with backslash
                    file write mktbl "`: word `col' of `: colnames matrix'' \ /// " _n
                }                
            }

            if `row'>1 & `row'<`rows'{ // write info for every row but the last

                local rowd=`row'-1
                local cold=`col'-1

                if `col'==1 {            // first col is rowname
                    file write mktbl "`: word `rowd' of `: rownames matrix'', "
                }

                if `col'>1 & `col'<=`colmax' { // write info with comma
                    file write mktbl " c`rowd'_`cold', "            
                }    

                if `col'==`cols' { // write info with backslash
                    file write mktbl " c`rowd'_`cold' \ /// " _n
                }                
            }

            if `row'>1 & `row'==`rows'{ // write info for the last row

                local rowd=`row'-1
                local cold=`col'-1

                if `col'==1 {            // first col is rowname
                    file write mktbl "`: word `rowd' of `: rownames matrix'', "
                }

                if `col'>1 & `col'<=`colmax' { //  write info with comma
                    file write mktbl " c`rowd'_`cold', "            
                }    

                if `col'==`cols' { // write info with backslash
                    file write mktbl " c`rowd'_`cold' ) " _n
                }                
            }
        }
    }
file close mktbl    
}

* to call the now ready dofile, we put the logging back on
//ON

log query _all

/**/do $odir/marktable.do

When i run the code, MarkDoc tells me that the log file is off, although the output of the log query _all tells me otherwise.

What am i missing here?
Or is there another (simpler) way to get nice tables in MarkDoc?

[haghish]

Sebastian Pehle

Interesting idea and I hope to try your command/package soon.

The problem you are running into is not caused by markdoc. but tbl command can cause that warning because you are naming your log file which makes accessing the log file more complicated.

For this, tbl only checks for the current log using which does not reveal the status of the log files that you have named. Therefore, in your example, Stata will return "closed".

 log query
(closed)

However, MarkDoc does not care whether your log file is named or not. As long as you pass it the name or path to the log file, it renders it.

Regarding making interesting tables, the best approach would be to use the existing packages and export a LaTeX / HTML / Markdown table. Then you can simply use the

//IMPORT filename

syntax and load it in your document automatically. There are already LaTeX and HTML packages that create tables. But I haven't seen a Markdown table generator. If you are interested to program such a package, you can just focus on generating a Markdown table file. It makes your life much easier than generating the Markdown code for tbl command.

Another solution would be simply using the pandoc command, provided by MarkDoc to convert a LaTeX or HTML table (generated by other packages) to MarkDown and then importing it to MarkDoc. So basically, you can write a program/package that:

1. uses existing packages for exporting HTML or LaTeX tables
2. use the pandoc command provided by MarkDoc to convert it to Markdown and return a file!

That itself is a contribution if you can make it work for all of the existing packages. Anyway, I just wanted to point out some workarounds.

[haghish]

I just forgot to mention that it is "just a warning"...

[Sebastian Pehle]

haghish

Thanks for your reply. Just to proof that my idea would work im now at the following point:
I got an ado for converting my table to a matrix and an ado for converting the matrix to a tbl command.

With tabmat varlist one can get a matrix from a one way or two way table.

* tabmat.ado
* extracts frequencies of the tab command and writes value labels as
* rownames
********************************************************************************

capture program drop tabmat
program define tabmat
version 12
local varlist = "`0'"

* count the variables
local nw : word count `varlist'
* lookup the variable names
local fw : word 1 of `varlist'
local sw : word 2 of `varlist'

* if there are no variables to tab
if `nw'==0 {
    dis "no variables defined"
}

* if there is just one variable, there are no other colnames than n
if `nw'==1 {
    quietly tab `varlist',matcell(tabmat)

    quietly levelsof `varlist', local(levels)     // extract levels of variable
    local lbe : value label `varlist'             // extract name of value label
    foreach l of local levels {
        local f`l' : label `lbe' `l'             // extract value label for each value
    }

    foreach l of local levels {
        local f_all = "`f_all' `f`l''"             // "add" all valuelabels
    }

    mat rownames tabmat=`f_all' // define rownames for matrix
    mat colnames tabmat= n      // define colnames for matrix
}

* if there are two variables, include row and colnames
if `nw'==2 {
    * two variables

    quietly tab `varlist',matcell(tabmat)

    foreach var of local varlist {
        quietly levelsof `var', local(levels) // extract levels of variable
        dis "`levels'"
        local lbe : value label `var' // extract name of value label
        // mac dir

        if "`lbe'"!="" { // if the variable has value labels extract them
            foreach l of local levels {
                local f`l' : label `lbe' `l'
            }            
        }        

        foreach l of local levels { // add the valuelabels all together
            local f_all`var' = "`f_all`var'' `f`l''" // "add" all valuelabels
        }    

        if "`lbe'"=="" { // if the variable has no value labels, print values
                local f_all`var'=""                
            foreach l of local levels {
                local f_all`var' = "`f_all`var'' `l' " // "add" all values
            }            
        }
    // local lbe=""
    }

    mat rownames tabmat=`f_all`fw'' // define rownames for matrix
    mat colnames tabmat=`f_all`sw'' // define colnames for matrix
}


if `nw'>2 {
    dis "too many variables defined"
}

end

With mattbl one can get the tbl command in the log, but only after commenting out the warning in your tble ado, because the warning is shown in the log otherwise

* mattbl: convert a matrix to a tbl(e) command
************************************************************************
capture program drop mattbl
program define mattbl
version 12
local matname = "`0'"

local rowt=""
local cols=colsof(`matname')
local rows=rowsof(`matname')

local ++cols
local ++rows


* Writing a program thats generating the tble code *****************************
quietly file open mktbl`matname' using $odir/marktable`matname'.ado, write replace
quietly {
  file write mktbl`matname' "capture program drop mattbl_dofile_gen" _n  
  file write mktbl`matname' "program define mattbl_dofile_gen" _n  
  file write mktbl`matname' "version 12" _n  
  forvalues row=1/`rows' {    // every row of matrix

    forvalues col=1/`cols' {  // every column of matrix

      local colmax=`cols'-1   // local for last col ->put \ in there<-    
    
      if `row'==1 {       // put rownames in the first row

        if `col'==1 {
          file write mktbl`matname' "noi tble ( . ,  /// " _n // write tbl intro
        }
        
        if `col'<=`colmax' { // write info with comma
          file write mktbl`matname' "`: word `col' of `: colnames `matname''', "
        }

        if `col'==`cols' { // write info with backslash
          file write mktbl`matname' "`: word `col' of `: colnames `matname''' \ /// " _n
        }       
      }

      if `row'>1 & `row'<`rows'{ // write info for every row but the last

        local rowd=`row'-1
        local cold=`col'-1

        if `col'==1 {       // first col is rowname
          file write mktbl`matname' "`: word `rowd' of `: rownames `matname''', "
        }

        if `col'>1 & `col'<=`colmax' { // write info with comma
          local mentry=`matname'[`rowd',`cold']
          file write mktbl`matname' " `mentry', "      
        }

        if `col'==`cols' { // write info with backslash
          local mentry=`matname'[`rowd',`cold']          
          file write mktbl`matname' " `mentry' \ /// " _n
        }       
      }

      if `row'>1 & `row'==`rows'{ // write info for the last row

        local rowd=`row'-1
        local cold=`col'-1

        if `col'==1 {       // first col is rowname
          file write mktbl`matname' "`: word `rowd' of `: rownames `matname''', "
        }

        if `col'>1 & `col'<=`colmax' { //  write info with comma
          local mentry=`matname'[`rowd',`cold']                    
          file write mktbl`matname' " `mentry', "      
        }

        if `col'==`cols' { // write info with backslash
          local mentry=`matname'[`rowd',`cold']                    
          file write mktbl`matname' " `mentry' ) " _n
        }       
      }
    }
  }
  file write mktbl`matname' "end" _n  
}
quietly file close mktbl`matname'  
* dofile done ******************************************************************

* quietly execute the dofile
quietly do $odir/marktable`matname'.ado

* run the created command by calling the program
mattbl_dofile_gen

* end the program
end
********************************************************************************

And if everythings right, one could process a document.

* Output Directory
global odir "youroutputdir"


**** begin markdoc log**********************************************************
cap log close _all
quietly log using $odir/statalist_exmpl.smcl, replace smcl name(statalist)

**** Write Introduction

/***
# Tables with matrices in Markdoc

To get some nice looking tables in docx, you could convert frequencies to
matrices and use the tble command for a nicer output.

***/

* We dont want the following in the log, so we turn it off

//OFF

sysuse auto.dta
tabmat foreign

//ON

/***
## Cars
In the Data there are modern and foreign cars that have to be repaired.
***/

/**/ mattbl tabmat

//OFF

/**/ tabmat foreign rep78

//ON

/***
You can get crosstables as well. The second Var has no valuelabels, so the
values itself are the categories.
***/

/**/ mattbl tabmat

//OFF

label define repl 1 "1_time" 2 "2_times" 3 "3_times" 4 "4_times" 5 "5_times"
label values rep78 repl

/**/ tabmat foreign rep78

//ON

/***
Here again with defined value labels for the rep. As the program retrieves the
rownames of a matrix, spaces in the name are not allowed. I guess one could
alter it to retrieve the Value labels itself
***/

/**/ mattbl tabmat

/***
## Defined matrices and limitations
Or you could just create matrix yourself and print it...
***/

//OFF

matrix justdots=J(10,7,.)

//ON

/**/ mattbl justdots


/***
While the 10x7 matrix is just fine, a 10x8 matrix wont be processed.
Dont know yet why. Perhaps because some logging issue?
***/

//OFF

matrix justdots2=J(10,8,.)

//ON

/**/ mattbl justdots2

// cap erase $odir/marktable.do

qui log close statalist

cd $odir

markdoc statalist_exmpl.smcl, replace export(pdf) install title("Statalist Example")

and would get something thats quite nice, but is no one stop shop for tables/matrices/whatever.

I guess ill leave this aside for some time now and play around with your suggestion of using existing procedures for generating the table and then reimport it into the to be processed document.

[haghish]

Sebastian Pehle

1. It looks pretty neat I look forward to read the documentations and test it.
2. You can definitely make this in 1 step i.e. the other functions should be called inside the main command. Why not calling the other functions inside mattbl?
3. There is no longer a tble.ado. It was renamed to tbl.ado several months ago. But the package has only been updated on GitHub.
4. I removed the warning for the tbl command. You can reinstall Weaver package from https://github.com/haghish/Weaver
5. It would be much easier to answer your questions if you publish your program so that people can try your command... If you don't want to release it on SSC yet, just release it on GitHub or send a pull-request to Weaver package, which includes the tbl