Python Pipenv: A Better Package Manager

Many people prefer a Python tool called pipenv instead of using pip install and virtualenv separately. It’s up to you to decide which method you like better. Perhaps you work in a team where pip + virtualenv is the norm, that’s OK. Maybe, after reading this, you want to convince them to use pipenv instead!

Pipenv vs virtualenv

Using pipenv as a Python package manager has a number of advantages compared to using pip and virtualenv separately. These are the main ones:

• You no longer need to use pip and virtualenv separately. Instead, you have one tool that does it all, and more!
• pipenv separates your top-level dependencies from the last tested combination (e.g., the pip freeze output). This makes dependency management more user-friendly for you as a developer.
• pipenv encourages the use of the latest versions of dependencies to minimize security risks. It can even scan your dependencies for known vulnerabilities.
• pipenv gives insight into your dependency graph with pipenv graph.
• pipenv hashes all dependencies. It will detect packages that have been tampered with after you initially included them as a dependency.
• pipenv can work with requirements.txt files too. If there is one, it will automatically detect it and convert it into a Pipfile.

Perhaps some of these points confuse you. Don’t worry, they will all be explained!

Pipenv install

Chances are you have Pip installed as a system-wide package already. I do this on my systems. Sure, I use virtual environments for all my projects. But I also installed a couple of tools like pip as system-wide packages. If you have pip installed, simply use it to install pipenv:

$ pip install --user pipenv

The --user option installs pipenv for the local user only. If you use it, you ensure that you won’t interfere with system-wide packages. Honestly, I just installed it globally without any problems. If you get an error, that pip was not found, try pip3.

If you are on MacOS or Linux, I can also recommend Homebrew. Go to the Homebrew website to install it. After that, it’s as simple as:

$ brew install pipenv

How to use Python pipenv

Now that you have pipenv installed, let’s try it!

First, create a directory. I called mine myenv. Now cd into the directory and install your first dependency with pipenv:

$ pipenv install requests

What happens next is:

• pipenv detects there’s no virtual environment yet, so it creates one.
• It installs requests.
• It creates two files: Pipfile and Pipfile.lock.

The screenshot in below shows pipenv in action on my system.

After it’s done, you can optionally enter the virtual environment with:

$ pipenv shell

This is optional because you can also launch a command in the virtual environment like this:

$ pipenv run <your command>

For example, to start a Python script inside the virtual environment:

$ pipenv run python3 myscript.py

How does Pipenv work?

I don’t like black boxes. I want to know what’s going on. So let’s first inspect Pipfile. It should look like this:

[[source]]
 name = "pypi"
 url = "https://pypi.org/simple"
 verify_ssl = true

 [dev-packages]

 [packages]
 requests = "*"

 [requires]
 python_version = "3.6"

As you can see, Pipfile contains our top-level requirements without specifying a version number because we didn’t specify it. Now take a look at Pipfile.lock. It’s a big file, so I’ll just show a snippet here to illustrate:

...
"requests": {
  "hashes": [
    "sha256:43999036bfa82904....",
    "sha256:b3f43d496c6da..."
  ],
  "index": "pypi",
  "version": "==2.23.0"
},
...

Requests and all of its dependencies are listed, including the version numbers. Basically, this is an advanced version of the output of pip freeze. These are the versions we worked with. It’s the combination of packages that are guaranteed to work.

But where’s the virtual environment? If you look closely at the screenshot in @fig:pipenv_in_action, you see the virtualenv is created in ~/.local/share/virtualenvs/myenv-mAvwj65b/. We could have asked pipenv too with pipenv --venv.

Since we are already familiar with the structure of a virtual environment, you’ll notice that this one is no different. It’s just located somewhere else, outside of your project structure. This way, you don’t need to exclude it from your version control system too.

If you create another virtual environment with the same name in another location, it won’t be reused. The hash in the directory name is, I presume, based on the path to your virtualenv. Pipenv uses it to find the correct one despite double names.

Separating Development Packages

With pip, you are only able to separate development requirements from production requirements by using a second requirements file (e.g. requirements-dev.txt). With pipenv, you can simply use the --dev option:

$ pipenv install pytest --dev

Another developer that starts working on your project can install all requirements, including the developer requirements, using:

$ pipenv install --dev

Dependency Management

pipenv attempts to install all sub-dependencies required by your core dependencies. Conflicts might arise, though. If package A requires at least version 1.10 of package B, while package C requires version 1.9, you have a problem. pipenv can’t solve it for you, but it will warn you about it and refuse to continue by default. It also offers a nice insight into your dependencies with the pipenv graph command:

$ pipenv graph
pytest==5.3.5
  - attrs [required: >=17.4.0, installed: 19.3.0]
  - importlib-metadata [required: >=0.12, installed: 1.5.0]
    - zipp [required: >=0.5, installed: 3.1.0]
  - more-itertools [required: >=4.0.0, installed: 8.2.0]
  - packaging [required: Any, installed: 20.3]
    - pyparsing [required: >=2.0.2, installed: 2.4.6]
    - six [required: Any, installed: 1.14.0]
  - pluggy [required: >=0.12,<1.0, installed: 0.13.1]
    - importlib-metadata [required: >=0.12, installed: 1.5.0]
      - zipp [required: >=0.5, installed: 3.1.0]
  - py [required: >=1.5.0, installed: 1.8.1]
  - wcwidth [required: Any, installed: 0.1.8]
requests==2.23.0
  - certifi [required: >=2017.4.17, installed: 2019.11.28]
  - chardet [required: >=3.0.2,<4, installed: 3.0.4]
  - idna [required: >=2.5,<3, installed: 2.9]
  - urllib3 [required: >=1.21.1,<1.26,!=1.25.1,!=1.25.0, installed: 1.25.8]

Detection of Security Vulnerabilities

pipenv can scan your packages for security vulnerabilities. It does so by using the safety package. Enter:

$ pipenv check

and pipenv will scan your dependency graph for packages with known security vulnerabilities!