PATH and Login bash on MacOS

It took a while to run down why certain older versions of programs were always launched when I was in Visual Studio Code, but not when I opened a terminal window. I first found this in relationship to nvm and documented it in this post: VS Code and the Node Version Manager. Unfortunately it affects other commands too, and it isn't just a VS Code issue, so I thought a more general post was in order.

Of course we know that the PATH environment variable is a colon-separated list of directories that are searched sequentially and the first match for a command is used:

$ echo $PATH

/usr/local/Cellar/python/3.7.4/bin:/usr/bin:/usr/local/bin

So in ~/.bash_profile we make sure that we set up the path in the correct order. For example, there is a python program in /usr/bin, so if we install a new python version we make sure the PATH points to the new installation before it points to /usr/bin. Python 3.7 has to come before /usr/bin in the PATH!

Some folks like to put PATH modifications in ~/.bashrc which is fine. You just have to remember that ~/.bashrc is supposed to run for every bash instance, so don't keep extending the PATH every time it gets called. This post explains it: Always Modify PATH and Other Variables Conditionally.

The problem is that MacOS VS Code (and other tools) may launch bash with a -l  or --login option. This creates a problem on MacOS because the bash program has been altered so that -l forces /usr/bin and /usr/local/bin to the front of the PATH. It is actually a really good security feature to make sure that built-in programs are located before a Trojan horse. Unfortunately sometimes we need to override that. Specifically, three important things that happen when -l is used:

1. --login keeps the existing environment, but re-sources the ~/.bash_profile. If this file appends to the PATH variable, the PATH gets doubled-up. Changes in the ~/.bash_profile have to be carefully set up so that if the PATH already contains what is needed, it is not blindly appended to the PATH again.
2. --login does not source ~/.bashrc, so ~/.bash_profile needs to source ~/.bashrc if aliases and other functions are defined there.
3. --login in some versions of bash rearranges the PATH variable! If anything was placed in the PATH in front of the system defined /usr/bin:/usr/sbin:/bin:/sbin:/usr/local/bin, then that is moved to the end of the PATH. And, in this case that means that /usr/local/bin/node gets found before the node executable that nvm expects in its own directories. And that means the error up above gets found.

This is what our path looks like after bash -l is done with it:

$ bash -l

$ echo $PATH

/usr/bin:/usr/local/bin:/usr/local/Cellar/python/3.7.4/bin

So the configuration to fix this needs to follow these steps:

1. Not duplicating values in ~/.bash_profile is right out of the Always Modify PATH and Other Variables Conditionally post. Here I am looking for the path to python:



dir=/usr/local/Cellar/python/3.7.4/bin
echo $PATH | grep -v -q python
then
    export PATH=$PATH:$dir
else
    export PATH=$(echo$PATH | sed -E "s|(.*):?([^:]*python[^:]*):?(.*$)|\1:$DIR:\3|" | sed -E "s|::|:|g" | sed -E "s|:$||")
fi 



2. Then source ~/.bashrc at the end of the ~/.bash_profile to make sure it is loaded for "login" shells. We have to fix the PATH in ~/.bashrc, which runs after bash munges the PATH and moved /usr/bin and /usr/local/bin:



source ~/.bashrc




3. For each command that has to come before /usr/bin in PATH, e.g. python, use the sed pipeline to rearrange the order of the path. It's similar to the sed up above, just the output order of the directories is reversed:



export PATH=$(echo $PATH | sed -E "s|(.*):?([^:]*python[^:]*):?(.*$)|\2:\1:\3|" | sed -E "s|::|:|g" | sed -E "s|:$||")

This will provide a stable configuration that handles these special cases. Not just inside VS Code, but for all other terminal windows as well.