Python - Can't find PIP on Mac

[Joe Prendergast]

Hello,

I'm trying to run Python Spyder 3.3.6 from Stata. Stata can't find PIP. The "Which PIP" command can't locate it. I'm using a Mac. Does anyone know what the path is? Or what the name of the script file is?

Thanks,
Joe

[William Lisowski]

Perhaps the discussions at these earlier topics will be helpful

https://www.statalist.org/forums/for...le-not-working

https://www.statalist.org/forums/for...shared-library

[Joe Prendergast]

Thanks William. I've been able to run Python from Stata. I just can't import modules like numpy. I thing if I knew where the PIP script file was I could do this, But it doesn't seem to be located in any of the paths that the topics you linked me to show how to list. I think my only hope is to find the name of the PIP script file and search for it.

[William Lisowski]

Here's a better link - a Stata FAQ that includes a link described as instructions for installing PIP if your installation lacks it. Your problem seems to be that it wasn't until the 3.4 release of Python 3 that PIP was included in the installation.

https://www.stata.com/support/faqs/p...-party-python/

[Joe Prendergast]

I'm finding promising looking directories like:

/users/jprenderg/opt/anaconda3/lib/python3.7/site-packages/pip/

/users/jprenderg/opt/anaconda3/lib/python3.7/site-packages/numpy/

/users/jprenderg/opt/anaconda3/lib/python3.7/site-packages/scipy/


But when is do something in Stata like:

python set userpath "/users/jprenderg/opt/anaconda3/lib/python3.7/site-packages/numpy"
although the command executes, I still can't import the package.

[Joseph Luchman]

Hi Joe,

When you do -python query- and -python search- what are the results you get?

For instance, when I run both I get:

. python query
----------------------------------------------------------------------------------------------------------------------------------------------------------
    Python Settings
      set python_exec      C:\Users\jluchman\AppData\Local\Programs\Python\Python38\python.exe
      set python_userpath  

    Python system information
      initialized          yes
      version              3.8.2
      architecture         64-bit
      library path         C:\Users\jluchman\AppData\Local\Programs\Python\Python38\python38.dll

. python search
----------------------------------------------------------------------------------------------------------------------------------------------------------
 Python environments found:  
 C:\Users\jluchman\AppData\Local\Programs\Python\Python38\python.exe
 C:\Users\jluchman\AppData\Local\Programs\Python\Python36\python.exe
 C:\Users\jluchman\Anaconda3\python.exe
----------------------------------------------------------------------------------------------------------------------------------------------------------

Which shows I have a few different versions of Python on my machine and that the one Stata is configured to call from is the non-Anaconda version 3.8.2.

I suspect you may have a similar situation where, if you were to configure Stata to call the Anaconda versions, you would be able to load -numpy-.

For example, Anaconda versions of Python come bundled with -scipy- and the Anaconda distribution of Python can find it like below:

. python set exec "C:\Users\jluchman\Anaconda3\python.exe"

. python
----------------------------------------------- python (type end to exit) --------------------------------------------------------------------------------
>>> import scipy
>>> end
---------

That code executed without error.

By contrast, I have not -pip install-ed -scipy- with my non-Anaconda version of Python that Stata and it throws an error.

. python set exec "C:\Users\jluchman\AppData\Local\Programs\Python\Python38\python.exe"

. python
----------------------------------------------- python (type end to exit) --------------------------------------------------------------------------------
>>> import scipy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'scipy'
r(7102);
>>> end
----------------------------------------------------------------------------------------------------------------------------------------------------------

Again, I suspect this may be the case on your machine too. That you have -numpy- as you have shown, but the Python version configured to be called by Stata does not have -numpy- associated with it.

- joe

[Joe Prendergast]

Hi Joe,

Thanks for all the info. Here are my results when I do what you did:


. python query
------------------------------------------------------------------------------------------------------------------------------------
Python Settings
set python_exec /usr/local/bin/python3
set python_userpath /users/jprenderg/opt/anaconda3/lib/python3.7/site-packages/pip/_vendor

Python system information
initialized yes
version 3.8.2
architecture 64-bit
library path /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/config-3.8-darwin/libpython3.8.dylib

. python search
------------------------------------------------------------------------------------------------------------------------------------
Python environments found:
/usr/bin/python
/usr/local/bin/python3
------------------------------------------------------------------------------------------------------------------------------------

It looks like my Stata is configured for the non-Anaconda version too. When I do "python query" I get two paths but no python.exe file to use for "python set exec". In fact, I can't even find:

/usr/bin/python
/usr/local/bin/python3

on my machine. I did search in finder for python.exe files and can't find any. This may be complicated by the fact that I'm using a Mac...

[Joe Prendergast]

I think the basic question is why isn't Stata's "python search" finding:

/Users/jprenderg/opt/anaconda3/bin

The directory exists and $PATH finds in when I execute the command on the MAC Terminal

[William Lisowski]

I think that is a question for Stata Technical Services.

I'd guess that the installation into the opt directory (which on my system does not exist) may have befuddled Stata.

Assuming Stata launches a command to search $PATH for python, it's possible the $PATH seen by the command does have the $PATH you think it does. For example, on my system, running the macOS Terminal, if from the Shell menu I choose New Command ... and run "zsh -c set" leaving the "run within a shell" box unchecked, the PATH it displays in not my full PATH. But if I check that box, whatever shell it's run within picks up my full PATH and that is passed to zsh.

I'm not saying Stata does this, but it is a demonstration that the PATH is not always what one might expect, because of the multiplicity of ways in which it can be set (or left unchanged).

[Joe Prendergast]

Thanks William,

I haven't had much luck with Stata technical services. They just keep sending me FAQ links.

This helps though. It's now just a matter of hunting for the right path. Doesn't seem like there are any .exe files, just .app files. I'll keep looking.

Joe

[Joe Prendergast]

Finally got it:

set python_exec "/users/jprenderg/opt/anaconda3/bin/python"

This also sets the correct library path:

/Users/jprenderg/opt/anaconda3/lib/libpython3.7m.dylib

Thanks for all the help!

Joe

[William Lisowski]

From post #10

Doesn't seem like there are any .exe files, just .app files.

My apologies, I see I overlooked this mistake when you first made it in post #7. Sorry I didn't see this earlier.

As you've learned, macOS does not have .exe files. Applications may have an .app extension, when extensions are being displayed. But an extension is not required for a file to be "executable".

On my macOS system with the included installation of python (no anaconda, etc), this output from a Terminal window shows us that the executable for "python" is ultimately a file named "python2.7" and the executable for python3 is an excutable named "python3". No .exe, no .app, befitting the underlying Unix-derived operating system.

lisowskiw 35% which python
/usr/bin/python
lisowskiw 36% ls -l /usr/bin/python
lrwxr-xr-x  1 root  wheel  75 Mar 21 15:44 /usr/bin/python -> ../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
lisowskiw 37% python

WARNING: Python 2.7 is not recommended.
This version is included in macOS for compatibility with legacy software.
Future versions of macOS will not include Python 2.7.
Instead, it is recommended that you transition to using 'python3' from within Terminal.

Python 2.7.16 (default, Feb 29 2020, 01:55:37)
[GCC 4.2.1 Compatible Apple LLVM 11.0.3 (clang-1103.0.29.20) (-macos10.15-objc- on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ^D

lisowskiw 38% which python3
/usr/bin/python3
lisowskiw 39% ls -l /usr/bin/python3
-rwxr-xr-x  1 root  wheel  31488 Mar 17 11:42 /usr/bin/python3
lisowskiw 40% python3
Python 3.7.3 (default, Apr  7 2020, 14:06:47)
[Clang 11.0.3 (clang-1103.0.32.59)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ^D

[Joe Prendergast]

No worries. You were a lot of help.

The "python" that worked is actually listed as filetype "alias" It has no extension.

[Joe Prendergast]

I can now import packages using code such as:

import scipy

But I can't import scipy sub-packages

For example:

from scipy import stats

or

import scipy.stats

return error messages. Any ideas? I can do this directly in Spyder just fine. Here is the error message when I run "from scipy import stats" in from stata:

Traceback (most recent call last):
File "/users/jprenderg/desktop/python/pyex2.py", line 9, in <module>
from scipy import stats
File "/users/jprenderg/opt/anaconda3/lib/python3.7/site-packages/scipy/stats/__init__.py", line 379,
> in <module>
from .stats import *
File "/users/jprenderg/opt/anaconda3/lib/python3.7/site-packages/scipy/stats/stats.py", line 180, in
> <module>
import scipy.special as special
File "/users/jprenderg/opt/anaconda3/lib/python3.7/site-packages/scipy/special/__init__.py", line 64
> 1, in <module>
from ._ufuncs import *
ImportError: dlopen(/users/jprenderg/opt/anaconda3/lib/python3.7/site-packages/scipy/special/_ufuncs.c
> python-37m-darwin.so, 2): Symbol not found: _main
Referenced from: /users/jprenderg/opt/anaconda3/lib/python3.7/site-packages/scipy/special/_ufuncs.cp
> ython-37m-darwin.so
Expected in: flat namespace
in /users/jprenderg/opt/anaconda3/lib/python3.7/site-packages/scipy/special/_ufuncs.cpython-37m-darwi
> n.so
failed to execute the specified Python script file
r(7103);

[Joe Prendergast]

I contacted Stata Tech Services about being able to import scipy but not scipy.stats. Here is their reply:

This is a known issue that only happens on Mac. This bug can be only reproduced
on some Macs that have a particular configuration- but we haven't been able
to track it down yetc. It seems that scipy uses some shared libraries that
Stata fails to load. Our developers are working to find a solution. Once we
get something, we will let you know.

There is a workaround for this suggested on github:

conda config --add channels conda-forge
conda install libgfortran=3.0.0=0

But I need pip to work to install conda on stata. The FAQ on pip gives a Windows path. I found the pip.py file on my mac at:

/users⁩/⁨jprenderg⁩/opt⁩/anaconda3⁩/⁨pkgs⁩/conda-4.7.12-py37_0⁩/⁨lib⁩/python3.7⁩/⁨site-packages⁩/⁨conda_env⁩/installers⁩/pip.py

But I can't run pip.py:

. python script /users⁩/⁨jprenderg⁩/opt⁩/anaconda3⁩/⁨pkgs⁩/conda-4.7.12-py37_0⁩/⁨lib⁩/python3.7⁩/⁨site-
> packages⁩/⁨conda_env⁩/installers⁩/pip.py
file /users⁩/⁨jprenderg⁩/opt⁩/anaconda3⁩/⁨pkgs⁩/conda-4.7.12-py37_0⁩/⁨lib⁩/python3.7⁩/⁨site-packages⁩/⁨
> conda_env⁩/installers⁩/pip.py not found

I moved pip.py to my own directory and this runs without errors:

python script /users/jprenderg/desktop/python/pip.py

But still Stata doesn't recognize the pip command:

. pip -V
command pip is unrecognized
r(199);

Anyone have any ideas on how to get Stata to run pip on a Mac