Getting started with Python on Ubuntu

A fresh installation of Ubuntu 14.04 LTS already has Python installed. The last time I checked, it was Python 3.4.3 and Python 2.7.6. That's great but for development purposes it's better for you and the system if you don't mess with the system's default Python.

The recommended approach is to set up isolated Python environments with venv or virtualenv. The Effective Python blog has a great article that goes in-depth into the reasons for doing it this way and so I'd only list the important points here:

  1. Large and complex programs often need to use external dependencies.
  2. By default, the recommended tool for installing Python packages, pip, installs them in a global location.
  3. This can lead to a problem when you have transitive dependencies, i.e. the packages that the packages you install depend on. See an example here.
  4. Issues may also arise when collaborating with other developers as their version of Python and the global packages they have installed could be slightly different than your own.
  5. This can cause frustrating situations where a codebase works perfectly on one programmer's machine and is completely broken on another's.

In summary, on Ubuntu you already have Python installed but it's not fit for a professional development workflow. And that's why in this post I will show you the following:

  1. How to install the latest version of Python 3 and Python 2 from source, and
  2. How to use venv in Python 3, virtualenv in Python 2 and pip to isolate your dependencies.

Install Python 3 from source

As of this writing (Nov 19th, 2016) the latest stable Python 3 release is version 3.5.2. You can manually download its source from here but I'd show you via the command line as it's faster and easier to automate.

$ cd /tmp
$ wget -O- https://www.python.org/ftp/python/3.5.2/Python-3.5.2.tgz | tar xz
$ cd Python-3.5.2

I'd recommend that you take some time to explore the source directory, /tmp/Python-3.5.2. In particular, you can

  1. Read the README for general help on the process, and
  2. Read ./configure --help for help on the configuration step.

Next, let's install all the required and optional dependencies.

$ sudo apt-get update

# Required dependencies
$ sudo apt-get install build-essential

# Optional dependencies
$ sudo apt-get install libbz2-dev libgdbm-dev libsqlite3-dev libreadline6-dev libncurses5-dev libssl-dev zlib1g-dev liblzma-dev tk-dev

You're now ready to configure, build and install Python 3.

$ mkdir -p ~/local/python-3.5.2
$ ./configure --prefix=$HOME/local/python-3.5.2
$ make

# Optional step
$ make test

$ make install

Finally, run the interpreter to ensure it was installed correctly.

$ ~/local/python-3.5.2/bin/python3
Python 3.5.2 (default, Nov 19 2016, 08:07:39)  
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.  
>>> print('Hello, world!')
Hello, world!  
>>>

Use venv and pip to isolate dependencies

Python 3 (since version 3.3) comes with a module for creating virtual environments called venv. Each virtual environment has its own Python binary and can have its own independent set of installed Python packages.

Here's how you'd create, activate and use a virtual environment:

$ mkdir ~/myproject && cd ~/myproject

# Create an environment named venv
$ ~/local/python-3.5.2/bin/python3 -m venv venv

# Activate it with
$ source venv/bin/activate
# or
$ . venv/bin/activate

# Check the version of Python installed in the environment
(venv) $ python --version
Python 3.5.2  

Within the virtual environment you have access to pip which is the recommended tool for installing Python packages.

# Upgrade pip and setuptools as they may not be the latest
(venv) $ pip install --upgrade pip setuptools

# Install a package
(venv) $ pip install django

# Work, work, work
...

# Deactivate when done
(venv) $ deactivate

Read here to learn more about using pip.

That's it. You're all set to begin developing awesome libraries/applications in Python 3.

skip to the bonus section

Install Python 2 from source

As of this writing (Nov 19th, 2016) the latest stable Python 2 release is version 2.7.12. You can manually download its source from here but I'd show you via the command line as it's faster and easier to automate.

$ cd /tmp
$ wget -O- https://www.python.org/ftp/python/2.7.12/Python-2.7.12.tgz | tar xz
$ cd Python-2.7.12

I'd recommend that you take some time to explore the source directory, /tmp/Python-2.7.12. In particular, you can

  1. Read the README for general help on the process, and
  2. Read ./configure --help for help on the configuration step.

Next, let's install all the required and optional dependencies.

$ sudo apt-get update

# Required dependencies
$ sudo apt-get install build-essential

# Optional dependencies
$ sudo apt-get install libdb-dev libsqlite3-dev libreadline6-dev libncurses5-dev libssl-dev libbz2-dev libgdbm-dev tk-dev zlib1g-dev

You're now ready to configure, build and install Python 2.

N.B. I'm using the --with-ensurepip=upgrade option (learn about ensurepip here) in order to bootstrap pip into the Python installation. If we don't do this now then we'd need to use get-pip.py afterwards to get pip into the Python installation.

$ mkdir -p ~/local/python-2.7.12
$ ./configure --prefix=$HOME/local/python-2.7.12 --with-ensurepip=upgrade
$ make

# Optional step
$ make test

$ make install

Run the interpreter to ensure it was installed correctly.

$ ~/local/python-2.7.12/bin/python
Python 2.7.12 (default, Nov 19 2016, 09:04:39)  
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.  
>>> print('Hello, world!')
Hello, world!  
>>>

Finally, check that you have access to pip.

$ ~/local/python-2.7.12/bin/pip --version
pip 8.1.1  

It may not be the latest version but you can easily upgrade it.

$ ~/local/python-2.7.12/bin/pip install --upgrade pip
$ ~/local/python-2.7.12/bin/pip --version
pip 9.0.1  

Use virtualenv and pip to isolate dependencies

In Python 2 you have to use virtualenv for creating isolated environments.

Let's install it.

$ ~/local/python-2.7.12/bin/pip install virtualenv

Now you can create, activate and use a virtual environment.

$ mkdir ~/myproject && cd ~/myproject

# Create an environment named venv
$ ~/local/python-2.7.12/bin/virtualenv venv

# Activate it with
$ source venv/bin/activate
# or
$ . venv/bin/activate

# Check the version of Python installed in the environment
(venv) $ python --version
Python 2.7.12

# Install a package
(venv) $ pip install django

# Work, work, work
...

# Deactivate when done
(venv) $ deactivate

Read here and here to learn more about using pip and virtualenv respectively.

That's it. You're all set to begin developing awesome libraries/applications in Python 2.

Bonus

For convenience, you can alias the commands to give your fingers a rest. Here are some suggestions:

Python 3

alias python3.5='$HOME/local/python-3.5.2/bin/python3'  
alias python3=python3.5  
alias pyvenv='python3 -m venv'  

Python 2

alias python2.7='$HOME/local/python-2.7.12/bin/python'  
alias python=python2.7  
alias virtualenv='$HOME/local/python-2.7.12/bin/virtualenv'  

Place those in your .bashrc and you'd be good to go.

Notes

My notes on how I figured out everything for this post.

Python 3

To figure out all the optional dependencies I built Python 3.5.2 on a fresh installation of Ubuntu.

Python build finished successfully!  
The necessary bits to build these optional modules were not found:  
_bz2                  _curses               _curses_panel  
_dbm                  _gdbm                 _lzma  
_sqlite3              _ssl                  _tkinter  
readline              zlib  
To find the necessary bits, look in setup.py in detect_modules() for the module's name.  

I then used Ubuntu Packages Search to determine which packages would satisfy each dependency.

_bz2: libbz2-dev  
_dbm & _gdbm: libgdbm-dev  
_sqlite3: libsqlite3-dev  
readline: libreadline6-dev  
_curses & _curses_panel: libncurses5-dev  
_ssl: libssl-dev  
zlib: zlib1g-dev  
_lzma: liblzma-dev  
_tkinter: tk-dev  

Python 2

To figure out all the optional dependencies I built Python 2.7.12 on a fresh installation of Ubuntu.

Python build finished, but the necessary bits to build these modules were not found:  
_bsddb             _curses            _curses_panel  
_sqlite3           _ssl               _tkinter  
bsddb185           bz2                dbm  
dl                 gdbm               imageop  
readline           sunaudiodev        zlib  
To find the necessary bits, look in setup.py in detect_modules() for the module's name.  

I then used Ubuntu Packages Search and here to determine which packages would satisfy each dependency.

_bsddb: libdb-dev  
_sqlite3: libsqlite3-dev  
bsddb185: Obsolete version of bdb  
dl: deprecated in 2.6  
readline: libreadline6-dev  
_curses & _curses_panel: libncurses5-dev  
_ssl: libssl-dev  
bz2: libbz2-dev  
gdbm & dbm: libgdbm-dev  
sunaudiodev: obsolete interface to some SparcStation device  
_tkinter: tk-dev  
imageop: deprecated in 2.6  
zlib: zlib1g-dev  

Resources

These resources were very helpful in guiding me in the right direction. I couldn't have figured it out without them.

P.S. I know how confusing it can be for someone new to this. So if any of the steps above aren't clear or you're stuck at any point then hit me up in the comments below and we'd figure it out together.