How to install different Python versions: pyenv

Sébastien Puyet

Sébastien Puyet

— 9 min read
How to install different Python versions: pyenv

Managing multiple Python versions on a Linux machine is a common challenge. Whether you're supporting legacy applications that require Python 2.7, modern web services running Python 3.11, or data pipelines built for Python 3.9, having different Python versions coexist without conflicts is essential.

pyenv is a lightweight Python version management tool that allows you to install, manage, and switch between multiple Python versions easily. Unlike system package managers or containerization solutions, pyenv operates at the user level and doesn't interfere with your system's default Python installation, making it ideal for environments that need to support multiple applications with different Python requirements.

What is pyenv

Your have a Django application that requires Python 3.9 for stability, a new FastAPI service built for Python 3.11's performance improvements, and a legacy data processing script that only works with Python 2.7. Installing these versions through your system package manager often leads to conflicts, broken dependencies, or the inability to install certain versions at all.

pyenv solves this problem by building Python versions from source and installing them in isolated directories under your home directory. Each installation is completely independent, allowing you to switch between versions instantly without affecting system Python or other applications.

The key advantages over alternatives are significant. Unlike conda, pyenv doesn't create a separate ecosystem with its own package management, you still use pip and standard Python tools. Compared to Docker, pyenv has minimal overhead and doesn't require containerization knowledge. System package managers like apt or yum often lag behind Python releases and may not offer the specific versions you need.

However, pyenv isn't always the right choice. In heavily containerized environments where each application runs in its own Docker container, managing Python versions at the container level might be more appropriate. Similarly, if your server only runs a single Python application, using the system Python version might be simpler and more maintainable.

Prerequisites and System Requirements

pyenv works on most Linux distributions, including Ubuntu, CentOS, RHEL, Debian, and their derivatives. The tool builds Python from source, so you'll need development tools and libraries installed.

For Ubuntu and Debian systems, install the required dependencies:

sudo apt update
sudo apt install -y make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev \
libffi-dev liblzma-dev

For CentOS, RHEL, and Fedora systems:

sudo yum groupinstall -y "Development Tools"
sudo yum install -y gcc openssl-devel bzip2-devel libffi-devel \
readline-devel sqlite-devel xz-devel

Each Python version requires approximately 100-200MB of disk space, so plan accordingly if you're installing multiple versions. pyenv works best with regular user accounts, avoid installing as root since this can create permission issues and security concerns.

Verify your system is ready by checking that essential tools are available:

gcc --version
make --version
curl --version

If any of these commands fail, check the dependency installation for your distribution.

Installing pyenv on Linux Servers

The recommended installation method uses the official installer script, which handles most configuration automatically:

curl https://pyenv.run | bash

This script downloads pyenv and its useful plugins, including pyenv-virtualenv for virtual environment management. After installation, you'll need to configure your shell to recognize pyenv commands.

Add these lines to your shell configuration file (~/.bashrc for bash, ~/.zshrc for zsh):

export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

Reload your shell configuration:

source ~/.bashrc

Verify the installation by checking the pyenv version:

pyenv --version

You should see output like pyenv 2.3.x. If you encounter a "command not found" error, double-check that you've added the PATH configuration correctly and restarted your shell session.

For manual installation, you can clone the repository directly:

git clone https://github.com/pyenv/pyenv.git ~/.pyenv

Then follow the same shell configuration steps above. The manual method gives you more control but requires managing updates yourself.

When setting up pyenv for multiple users, each user should install pyenv in their own home directory rather than attempting a system-wide installation. This approach prevents permission conflicts and allows users to manage their Python versions independently.

Essential pyenv Commands Every Admin Should Know

Start by exploring what Python versions are available for installation:

pyenv install --list

This command shows all available Python versions, including CPython, PyPy, and other implementations. The list is extensive, so filter for specific versions:

pyenv install --list | grep "3.11"

To install a specific Python version:

pyenv install 3.11.5

The installation process downloads source code and compiles Python, which can take several minutes. For production systems, consider using the optimization flag for better performance:

CONFIGURE_OPTS="--enable-optimizations" pyenv install 3.11.5

This flag enables Profile Guided Optimization (PGO), making Python run 10-15% faster at the cost of longer compilation time.

View all installed Python versions:

pyenv versions

The output shows available versions with an asterisk indicating the currently active version:

  system
* 3.9.18 (set by /home/admin/.pyenv/version)
  3.11.5

Set a global default Python version that applies system-wide for your user:

pyenv global 3.11.5

For project-specific versions, use the local command within your project directory:

cd /var/www/myapp
pyenv local 3.9.18

This creates a .python-version file in the directory that automatically activates the specified Python version when you enter that directory.

Debug which Python executable is being used:

pyenv which python
pyenv which pip

These commands show the full path to the active Python and pip executables, helping troubleshoot version conflicts.

After installing new Python versions or packages with console scripts, run:

pyenv rehash

This command updates pyenv's shims to recognize new executables, ensuring commands like pip or flask work correctly.

Managing Python Versions

Developers often work on multiple applications with different Python requirements. Here's how to structure this effectively using pyenv's local version management.

Consider a server layout like this:

/home/bob/projects/
├── legacy-app/          # Requires Python 3.8
├── main-webapp/         # Requires Python 3.9
├── api-service/         # Requires Python 3.11
└── data-pipeline/       # Requires Python 3.10

Set up each application with its required Python version:

# Install required versions
pyenv install 3.8.18
pyenv install 3.9.18
pyenv install 3.10.13
pyenv install 3.11.5

# Configure each application
cd /home/bob/projects/legacy-app
pyenv local 3.8.18

cd /home/bob/projects/main-webapp
pyenv local 3.9.18

cd /home/bob/projects/api-service
pyenv local 3.11.5

cd /home/bob/projects/data-pipeline
pyenv local 3.10.13

Each directory now has a .python-version file that automatically activates the correct Python version when you enter that directory. This file should be committed to version control so team members and deployment processes use the same Python version.

Combine pyenv with virtual environments for complete isolation. While pyenv manages Python versions, virtual environments isolate package dependencies:

cd /home/bob/projects/main-webapp
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Alternatively, use the pyenv-virtualenv plugin for integrated virtual environment management:

pyenv virtualenv 3.9.18 webapp-env
pyenv local webapp-env

This approach creates a virtual environment tied to a specific Python version and automatically activates it when entering the directory.

Verify your setup by checking the Python version and package isolation in each directory:

cd /home/bob/projects/main-webapp
python --version  # Should show Python 3.9.18
pip list          # Shows only packages for this environment

cd /var/www/api-service
python --version  # Should show Python 3.11.5
pip list          # Shows different packages

Team Collaboration and Configuration Management

Standardizing Python versions across team members prevents the "works on my machine" problem. The .python-version file is your primary tool for this standardization.

Include .python-version files in your version control system:

# In your project root
echo "3.9.18" > .python-version
git add .python-version
git commit -m "Pin Python version to 3.9.18"

When team members clone the repository and have pyenv installed, they'll automatically use the correct Python version. Include setup instructions in your project's README:

# Development Setup
1. Install pyenv: `curl https://pyenv.run | bash`
2. Install project Python version: `pyenv install`
3. Create virtual environment: `python -m venv venv`
4. Activate environment: `source venv/bin/activate`
5. Install dependencies: `pip install -r requirements.txt`

For configuration management tools, create playbooks that install pyenv consistently. Here's an Ansible example:

- name: Install pyenv dependencies
  apt:
    name: "{{ pyenv_dependencies }}"
    state: present

- name: Install pyenv
  shell: curl https://pyenv.run | bash
  args:
    creates: "{{ ansible_env.HOME }}/.pyenv"

- name: Configure shell for pyenv
  lineinfile:
    path: "{{ ansible_env.HOME }}/.bashrc"
    line: "{{ item }}"
  loop:
    - 'export PYENV_ROOT="$HOME/.pyenv"'
    - 'export PATH="$PYENV_ROOT/bin:$PATH"'
    - 'eval "$(pyenv init -)"'

Document your team's Python version policy clearly. Specify which versions are approved for new projects, how often versions are updated, and the process for requesting new Python versions.

CI/CD Pipeline Integration

Integrating pyenv with CI/CD pipelines ensures your tests run against the same Python versions used in development and production.

For GitHub Actions, use the setup-python action combined with matrix builds to test multiple versions:

name: Test
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.9', '3.10', '3.11']
    
    steps:
    - uses: actions/checkout@v3
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v4
      with:
        python-version: ${{ matrix.python-version }}
    
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    
    - name: Run tests
      run: pytest

For Jenkins pipelines, install pyenv on your build agents and use it in your Jenkinsfile:

pipeline {
    agent any
    
    stages {
        stage('Setup Python') {
            steps {
                sh '''
                    export PYENV_ROOT="$HOME/.pyenv"
                    export PATH="$PYENV_ROOT/bin:$PATH"
                    eval "$(pyenv init -)"
                    pyenv install -s $(cat .python-version)
                    pyenv local $(cat .python-version)
                '''
            }
        }
        
        stage('Test') {
            steps {
                sh '''
                    export PYENV_ROOT="$HOME/.pyenv"
                    export PATH="$PYENV_ROOT/bin:$PATH"
                    eval "$(pyenv init -)"
                    python -m pip install -r requirements.txt
                    python -m pytest
                '''
            }
        }
    }
}

In containerized CI environments, consider whether pyenv adds value. If your production environment uses containers with specific Python versions, testing against those same container images might be more appropriate than using pyenv in CI.

Cache Python installations to speed up builds. In GitHub Actions:

- name: Cache Python installation
  uses: actions/cache@v3
  with:
    path: ~/.pyenv
    key: ${{ runner.os }}-pyenv-${{ hashFiles('.python-version') }}

Production Deployment Considerations

Using pyenv in production requires careful consideration of performance, security, and maintenance implications.

pyenv adds minimal runtime overhead since it uses simple shell scripts and symlinks to manage Python versions. However, the initial compilation process means longer deployment times if you're building Python versions on production servers. Consider pre-building Python versions on build servers and deploying the compiled binaries.

Security considerations include keeping pyenv itself updated and monitoring for security updates to Python versions. Since pyenv builds from source, you're responsible for applying security patches by installing updated Python versions:

# Check for updates
pyenv update

# Install security update
pyenv install 3.9.19
pyenv global 3.9.19

# Remove old version
pyenv uninstall 3.9.18

Monitor which Python versions are active across your services. Create a simple monitoring script:

#!/bin/bash
echo "Python version usage across services:"
for app in /var/www/*/; do
    if [ -f "$app/.python-version" ]; then
        echo "$app: $(cat $app/.python-version)"
    fi
done

Plan rollback procedures for when Python version updates cause issues. Keep the previous Python version installed until you've verified the new version works correctly:

# Install new version alongside old
pyenv install 3.11.6
# Test thoroughly before switching
pyenv global 3.11.6
# Only remove old version after confirming stability
pyenv uninstall 3.11.5

Debug Common Issues

Build failures are the most common pyenv issues, usually due to missing dependencies. If Python compilation fails, check the build log carefully:

pyenv install -v 3.11.5

The -v flag provides verbose output showing exactly where the build fails. Common issues include missing SSL libraries (needed for pip to work with HTTPS) or readline libraries (needed for interactive Python sessions).

If pyenv commands aren't recognized, verify your shell configuration. Check that the pyenv initialization code is in your shell's configuration file and that you've restarted your shell session:

echo $PATH | grep pyenv
which pyenv

Version switching problems often occur when the .python-version file contains a version that isn't installed. Check the file contents:

cat .python-version
pyenv versions

If the version in the file isn't listed by pyenv versions, either install that version or update the file.

Performance issues can arise from having too many Python versions installed. Each version consumes disk space and can slow down pyenv's version resolution. Remove unused versions:

pyenv uninstall 3.8.10

For permission issues, ensure pyenv is installed in your home directory, not system-wide. If you see permission errors, check the ownership of ~/.pyenv:

ls -la ~/.pyenv
# Should be owned by your user, not root

Advanced Configuration and Optimization

Optimize Python builds for production use by setting build flags. The most important optimization enables PGO (Profile Guided Optimization):

export CONFIGURE_OPTS="--enable-optimizations"
pyenv install 3.11.5

For servers with limited resources, disable some build features to reduce compilation time:

export CONFIGURE_OPTS="--disable-test-modules"
pyenv install 3.11.5

Manage disk space by regularly cleaning up old Python versions and build artifacts:

# List versions by disk usage
du -sh ~/.pyenv/versions/*

# Remove unused versions
pyenv uninstall 3.9.16

# Clear build cache
rm -rf ~/.pyenv/cache/*

Keep pyenv updated to get the latest Python versions and bug fixes:

pyenv update

For automated maintenance, create a script that updates pyenv and checks for new Python versions:

#!/bin/bash
echo "Updating pyenv..."
pyenv update

echo "Available Python 3.11 versions:"
pyenv install --list | grep "3\.11\." | tail -5

Back up your pyenv configuration by including the .python-version files in your backup strategy. The actual Python installations can be recreated, but your version configuration should be preserved.

Read more