🗎 Temporary file with -tempfile-

[Sergiy Radyakin]

Dear All,

I am looking for a way to make use of temporary files with the putpdf image command. Unfortunately, I find no way to make them understand each other. If the image is saved to a tempfile, the putpdf image command aborts with an error that the file type is not recognized (apparently it naively looks by the extension, and not by the actual contents, so fails on *.tmp files).

Saving the file as "`tmp'.png" is subject to two problems:
1. files are no longer temporary - they will not be deleted by Stata's tempfile tracker after the end of the code scope.
2. collisions are possible (and as a matter of fact likely) with files generated in previous runs, or in other sessions.

If in doubt, refer to the manual: "tempfile assigns names to the specified local macro names that may be used as names for temporary files. When the program or do-file concludes, any datasets created with these assigned names are erased."

Here "with these" means "exactly with these", not "derived from these".

Clearly adding the replace option is not a solution, as it hides the problem out of view, but doesn't solve it, and potentially can lead to incorrect reporting.

I see the following four ways of solving:
1. convince Stata to generate a temporary filename with a particular extension, in which case that could have been an option of the tempfile command, but there is none.
2. request Stata to create a temporary folder (not file, but folder) name, in which case I can be confident that my code owns the folder and can use fixed names (this would also be useful for my another project as well).
3. teach the putpdf to be more sophisticated and look into the contents of the file to determine it's type, or allow the user to specify it directly as an option, putpdf image ....., as(png).
4. teach graph save to save directly to the pdf document (do not confuse with saving as a pdf file, that's different).

All of these require intervention of the developers. Any advice from the fellow coders is much appreciated.

Best, Sergiy Radyakin

[Leonardo Guizzetti]

Perhaps I am not understanding the nuances of the problem you are running into. I think one issue is that -tempfile- leads to creation of .tmp files (at least in Windows), and with these, Stata will only delete them if they are datasets (?). What if you use -tempname- to generate a name, save the image with the appropriate image-specific extension, and issue explicit -rm- statements of desired.

[Sergiy Radyakin]

That would be just as good as saving to "C:\picture.png".

In any case the strategy is worse than using the tempfile command. The command tempname issues a temporary name that could be used for naming a variable (and that doesn't coincide with earlier issued varnames). This means that :

1. two Statas running in parallel will give you the identical temporary name __000000, which will immediately cause collision. The files generated with tempfile have a mechanism of preventing this, though, imho, rather unreliable.
2. if tomorrow Stata's rules for naming variables change, and, for example, a slash or a colon symbol is permitted in the variable names, this will break down my program, since tempname is not supposed to be used for naming the temporary files.
3. and the last, but not least, if there is a separate problem in my code, the files will not be removed, meaning their temporary nature is not observed. I could of cause put some capture-s around trying to secure the cleanup. But often that's just not worth the effort.

For uncommented reasons tempname has been used even by Stata manufacturers for generation of the temporary file names. With predictable consequences: confusion about freduse.

So yes, it's the nuances that matter.

PS: You wrote "Stata will only delete them if they are datasets (?)."
No, Stata will delete them whatever they are. As a matter of fact using tempfile for storing data temporarily is nonsense in Stata 16+. Use frames for that to minimize the I/O traffic. You use temporary files now to stage your output for interaction with external systems. However, if you are using datasets that approach you computer's memory capacity, that could be still useful.

[Hua Peng (StataCorp)]

You may use undocumented Mata function pathchangesuffix() to change the suffix of a temp file:

. mata: mata which pathchangesuffix
  pathchangesuffix():  lmatabase

. tempfile f

. di "`f'"
C:\Users\hpeng\AppData\Local\Temp\ST_2cd8_000001.tmp


* last argument 1 : absolute path, 0 : file name only
. mata:(void)pathchangesuffix("`f'", "png", "f1", 1)

. di "`f1'"
C:\Users\hpeng\AppData\Local\Temp\ST_2cd8_000001.png

For the source code:

viewsource pathchangesuffix.mata

Note, since the filename is prefixed with process id of Stata instance, they will not run into conflict with other Stata instances. But you are responsible for the deletion of the file with "png" extension. So a

cap erase `"`f1'"'

after its use is required for not leaving it behind after Stata exits.

[Sergiy Radyakin]

Dear Hua Peng,

thank you very much for your advice. At the moment I ended up doing a low-tech:

            tempfile modelimg
            local modelimg `"`modelimg'.png"'

which doesn't introduce any additional dependency on Mata or undocumented features and has the same effect.

The only cleaner solution would be to involve Python code, but that will introduce an additional dependency of that being installed on the user's system, which is not prevalent so far.

All of the hypothetical workarounds 1..4 in the first post are useful, and not particularly costly to implement (imho). It would be great if some of them are included in the future development.

Sincerely, Sergiy Radyakin

[Zhao Xu (StataCorp)]

Sergiy Radyakin thanks for bringing this to our attention. This was fixed in the 30 Jun 2020 update that was released today.

From the whatsnew:

putpdf image filename and putpdf table tablename(i,j) = image(filename) could fail if filename
(the image to be added) did not have a known extension. This has been fixed.

Now putpdf will check the image file and determine if the image type is known and supported without using its extension.

Here is a quick example:

sysuse auto, clear
sc mpg price

putpdf clear
putpdf begin

tempfile aa
quietly graph export "`aa'", as(png) replace

putpdf paragraph
putpdf image "`aa'", linebreak

putpdf save putpdfimg, replace