Bug in Python indentation?

[Joseph Coveney]

My Python-containing ado programs that ran before now give an r(7102); error: IndentationError: expected an indented block . . .. I wrote them last year in Release 17.0, but I thought that I'd used them as recently as earlier this summer (i.e., in the current release) without any trouble. When I went to use one today I discovered this new (mis)behavior.

I can get simple function definitions to run by using a reverse solidus and semicolon fix-up as shown in this post from April this year. For example, this

*! indent_bug.ado
program define indent_bug
    version 18.0
    syntax

    python: test()
    
end

version 18.0

python:

def test():
    print("Here.")
    print(". . . and Here.")

end

gives the error, but this

*! indent_bugbreaks.ado
program define indent_bugbreaks
    version 18.0
    syntax

    python: test()

end

version 18.0

python:

def test(): \
    print("Here."); \
    print(". . . and Here.")

end

runs without error.

Unfortunately, slightly more complicated code, such as

import json
def search(keyword):
    with open("F:/" + keyword + ".json", "r") as f:
        response = json.load(f)
    return(response)

still gives an error (although a different one) even with the reverse solidus-semicolon workaround. (That code snippet works as expected in my Python environment. Excuse the unusual choice of names—it's a test harness for another function that will call search() to access a search engine API.)

I've attached three files, indent_bug.ado, indent_bugbreaks.ado and test.py,* whiich I called in sequence from the command line

indent_bug
indent_bugbreaks
python script test.py, global
python: test()

Only the first ado file shows the error. The resulting output is in the log file indent_bug.smcl, which is also attached.

Is anyone else experiencing this or is it something peculiar to my setup? Or am I misunderstanding something? I've Googled python indentation stata but haven't turned up much on the topic, excepting that thread linked to above.

* test.py contains the same code as in the auxiliary Python section of indent_bug.ado (shown in the first code block above). I've had to append a .txt extension to test.py in order to circumvent the forum's limitations on attachments' file-name extensions. To run the relevant line in the code just above, remove the second file name suffix (rename test.py.txt → test.py after downloading).

[Zhao Xu (StataCorp)]

Hi Joseph Coveney,

I ran your code in Stata 17 and Stata 18 without any problem. I am using Python 3.11. Here are the output in my end for both:

. indent_bug
Here.
. . . and Here.

.
. indent_bugbreaks
Here.
. . . and Here.

.
. python script test.py, global

.
. python: test()
Here.
. . . and Here.

For us to further investigate your problem, can you tell us which Python version you are using and what system you are running Stata?

[Joseph Coveney]

For us to further investigate your problem, can you tell us which Python version you are using and what system you are running Stata?

Thank you for your quick response. Yes, it's Python 3.12.0 on Windows 10 Pro, 64-bit, updated-to-current KB as of yesterday, Stata 18.0, MP4. Apologies for not mentioning at least the Python version.

[Martyn Sherriff]

For what it is worth I get the same error with Python 3.13 / Windows 11 / Stata 18 BE:

. python:
----------------------------------------------- python (type end to exit) ------------------------------------------------------------------------------------------------------------------
>>> 
>>> def test():
  File "<stdin>", line 1
    def test():
               ^
IndentationError: expected an indented block after function definition on line 1
(3 lines skipped)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
r(7102);

end of do-file

[Zhao Xu (StataCorp)]

Hi Joseph Coveney and Martyn Sherriff,

Thank you for bringing this to our attention.

We can reproduce the problem you mentioned when calling Python from within Stata with both Python 3.12 and 3.13. This happens when calling a compound statement, such as if, for, and def. In Python 3.12 and above, we noticed that they seem to have changed the behavior of some Python/C API functions that we rely on for parsing. We are still investigating it. Once we find some more information, we will post back.

At the same time, we tested with the latest 3.11 subversion 3.11.6 and found no problem.

[Joseph Coveney]

In Python 3.12 and above, we noticed that they seem to have changed the behavior of some Python/C API functions that we rely on for parsing. We are still investigating it. Once we find some more information, we will post back.

Thank you for the update. I'll await further developments.

[Joseph Coveney]

From the whatsnew list of yesterday's update, "10. Python code, when defined within python[:] and end that were run from the Command window or from a do-file with Python 3.12 or above and when there was a compound statement (such as for or def), failed to execute the corresponding Python code. This has been fixed."

Thanks for this. My ado programs run again.