Using Python 3 in virtualenv – Dev

The best answers to the question “Using Python 3 in virtualenv” in the category Dev.

QUESTION:

Using virtualenv, I run my projects with the default version of Python (2.7). On one project, I need to use Python 3.4.

I used brew install python3 to install it on my Mac. Now, how do I create a virtualenv that uses the new version?

e.g. sudo virtualenv envPython3

If I try:

virtualenv -p python3 test

I get:

Running virtualenv with interpreter /usr/local/bin/python3
Using base prefix '/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4'
New python executable in test/bin/python3.4
Also creating executable in test/bin/python
Failed to import the site module
Traceback (most recent call last):
  File "/Users/user/Documents/workspace/test/test/bin/../lib/python3.4/site.py", line 67, in <module>
    import os
  File "/Users/user/Documents/workspace/test/test/bin/../lib/python3.4/os.py", line 634, in <module>
    from _collections_abc import MutableMapping
ImportError: No module named '_collections_abc'
ERROR: The executable test/bin/python3.4 is not functioning
ERROR: It thinks sys.prefix is '/Users/user/Documents/workspace/test' (should be '/Users/user/Documents/workspace/test/test')
ERROR: virtualenv is not compatible with this system or executable

ANSWER:

Python 3 has a built-in support for virtual environments – venv. It might be better to use that instead. Referring to the docs:

Creation of virtual environments is done by executing the pyvenv
script:

pyvenv /path/to/new/virtual/environment

Update for Python 3.6 and newer:

As pawciobiel correctly comments, pyvenv is deprecated as of Python 3.6 and the new way is:

python3 -m venv /path/to/new/virtual/environment

ANSWER:

simply run

virtualenv -p python3 envname

Update after OP’s edit:

There was a bug in the OP’s version of virtualenv, as described here. The problem was fixed by running:

pip install --upgrade virtualenv

ANSWER:

Install prerequisites.

sudo apt-get install python3 python3-pip virtualenvwrapper

Create a Python3 based virtual environment. Optionally enable --system-site-packages flag.

mkvirtualenv -p /usr/bin/python3 <venv-name>

Set into the virtual environment.

workon <venv-name>

Install other requirements using pip package manager.

pip install -r requirements.txt
pip install <package_name>

When working on multiple python projects simultaneously it is usually recommended to install common packages like pdbpp globally and then reuse them in virtualenvs.

Using this technique saves a lot of time spent on fetching packages and installing them, apart from consuming minimal disk space and network bandwidth.

sudo -H pip3 -v install pdbpp
mkvirtualenv -p $(which python3) --system-site-packages <venv-name>

Django specific instructions

If there are a lot of system wide python packages then it is recommended to not use --system-site-packages flag especially during development since I have noticed that it slows down Django startup a lot. I presume Django environment initialisation is manually scanning and appending all site packages from the system path which might be the reason. Even python manage.py shell becomes very slow.

Having said that experiment which option works better. Might be safe to just skip --system-site-packages flag for Django projects.

ANSWER:

I’v tried pyenv and it’s very handy for switching python versions (global, local in folder or in the virtualenv):

brew install pyenv

then install Python version you want:

pyenv install 3.5.0

and simply create virtualenv with path to needed interpreter version:

virtualenv -p /Users/johnny/.pyenv/versions/3.5.0/bin/python3.5 myenv

That’s it, check the version:

. ./myenv/bin/activate && python -V

There are also plugin for pyenv pyenv-virtualenv but it didn’t work for me somehow.