Abstract
It is 2021 and we are all using or heard of package managers in Python, among which are Pipenv and Poetry. I also built a new package manager PDM to solve similar problems. There exist some comparisons between them around the community, but this article is not going to talk about the user interface or their versatility, it is going to focus on two important aspects: performance and correctness.
Setup
Pipenv: HEAD@275f7e151eb0aa17702215165a371df7da9ad476
Poetry: 1.1.5
PDM: HEAD@d72ba5d2ee0b0305f917d8739667ba78465c5cc8
Python version: 3.9.1
Performance
Dependency set(in poetry format):
代码语言：javascript复制[tool.poetry.dependencies]
python = "^3.9"
requests = { git = "https://github.com/psf/requests.git", tag = "v2.25.0" }
numpy = "^1.19"
[tool.poetry.dev-dependencies]
pytest = "^5.2"This includes a Git dependency and a heavy binary dependency. In the following result, the cost times are measured in seconds.
Unless explicitly given, the measurements are done against the install command.
Result
| Pipenv | Poetry | PDM |
|---|---|---|---|
Clean cache, no lockfile | 98 | 150 | 58 |
With cache, no lockfile | 117 | 66 | 28 |
Clean cache, reuse lockfile* | 128 | 145 | 35 |
With cache, reuse lockfile** | 145 | 50 | 16 |
*: Test with command poetry add click pipenv install --keep-outdated click pdm add click respectively.
**: Commands are the same as above.
Performance Review
- Pipenv has a problematic cache system, which slows down the performance with the existence of caches
- Poetry and PDM both benefit a lot from the caches, PDM takes even less time.
- Pipenv uses a very different mechanism to reuse the lock file — it runs full locking first then modifies the content of the old lock file, while PDM can reuse the pinned versions in the lock file. Poetry improves a little with the lock file existing.
Correctness
The goal of these 3 package managers is to produce a reproducible environment and they all try their best to make it work on cross-platforms and python versions. Let's see how it turns out.
Python Compatibility
The project file is designed as following(Poetry):
代码语言：javascript复制[tool.poetry.dependencies]
python = "^3.6||^2.7"And PDM:
代码语言：javascript复制[project]
requires-python = ">=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*"They both reference to the same range of Python versions to support. Although Pipenv doesn't support a Python version range constraint, I will also post its results here as a reference.
Now let's add one dependency pytest from a clean project. This dependency is chosen purposely because it has a different version set to support Python 2 and Python 3.
Result of Poetry:
代码语言：javascript复制$ poetry add pytest
...
Using version ^6.2.2 for pytest
Updating dependencies
Resolving dependencies...
SolverProblemError
The current project's Python requirement (>=2.7,<3.0 || >=3.6,<4.0) is not compatible with some of the required packages Python requirement:
- pytest requires Python >=3.6, so it will not be satisfied for Python >=2.7,<3.0
Because no versions of pytest match >6.2.2,<7.0.0
and pytest (6.2.2) requires Python >=3.6, pytest is forbidden.
So, because foo depends on pytest (^6.2.2), version solving failed.Lock failed as 6.2.2 was picked but it was not compatible with python 2.7. Anyhow Poetry shows a well human-readable error message telling people what happened and how to solve it.
Result of Pipenv
Depending on Python 3 or Python 2 is used to create the virtualenv(with --three/--two option), pytest is locked to 6.2.2 and 4.6.11 respectively. The lock file surely can't work on both Python 2 and Python 3 environment at the same time.
Result of PDM
代码语言：javascript复制$ pdm add pytest
Adding packages to default dependencies: pytest
✔
