Focusing on Poetic Dependencies

A build tool for managing package dependencies

Introduction

In a previous post, pip-tools were discussed as a build tool for managing package dependencies. This post will focus on a relative newcomer: Poetry.

Comparisons

Within an app managed by pip-tools, we will typically find 2 files per environment: a dependency file, e.g. requirements.in; and a managed lock file, e.g. requirements.txt. Any dependency updates must be defined in the appropriate dependency file, and all environments must be updated to ensure constraints are properly handled.

Poetry introduces developer-friendly CLI commands to help reduce this churn. A simple `poetry add [package]` is all that is needed to update both the dependency and lock files, as well as install the package into the local environment. Additionally, the use of environment-specific files is instead managed with a single project definition file: pyproject.toml (see [PEP 518].

Unlike pip-tools, Poetry is intended to be installed system-wide. Multiple installation methods can be found in the documentation. A valid installation can be confirmed by checking the tool’s version from the command line.

$ poetry --version
Poetry version 1.1.7

Initialization

New projects can be started with a Poetry-defined structure using the `poetry new` command. Existing projects can be initialized with a new pyproject.toml file using the `poetry init` command, which includes an interactive guide. To demonstrate the latter, let’s continue with the app created in the previous post.Confirm or update the default options, and define the dependencies interactively to allow Poetry to add the appropriate requirements to pyproject.toml. Recall: we need fastapi for our production environment, and fastapi[all] for development.

$ poetry init

This command will guide you through creating your pyproject.toml config.

Package name [beep]:
Version [0.1.0]:
Author [Nick Whitt]:
License []:
Compatible Python versions [^3.9]:

Would you like to define your main dependencies interactively? (yes/no) [yes]

Search for package to add (or leave blank to continue): fastapi

Enter the version constraint to require (or leave blank to use the latest version):
Using version ^0.66.0 for fastapi

Add a package:

Would you like to define your development dependencies interactively? (yes/no) [yes]
Search for package to add (or leave blank to continue): fastapi[all]

Enter the version constraint to require (or leave blank to use the latest version):
Using version ^0.66.0 for fastapi

Add a package:

Generated file


[tool.poetry]
name = "beep"
version = "0.1.0"
description = ""
authors = ["Nick Whitt"]

[tool.poetry.dependencies]
python = "^3.9"
fastapi = "^0.66.0"

[tool.poetry.dev-dependencies]
fastapi = {extras = ["all"], version = "^0.66.0"}

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"


Do you confirm generation? (yes/no) [yes]

With a valid pyproject.toml in place, run `poetry install` to have Poetry resolve all (including dev) dependencies and generate a poetry.lock file. The option to install with “no-dev” packages is also available.

$ poetry install
Updating dependencies
Resolving dependencies... (0.4s)

Writing lock file

No dependencies to install or update

Note: as we already have an up-to-date virtual environment defined, there is nothing new for Poetry to do.

Activated Shell

With Poetry installed, we can easily run commands within the local virtual environment with `poetry run`, even if that environment is not active.

$ python --version
Python 2.7.16
$ poetry run python --version
Python 3.9.6
$ source .venv/bin/activate
(.venv) $ python --version
Python 3.9.6

If desired, you can just as easily start a new shell within the virtual environment with `poetry shell`.

$ poetry shell
Spawning shell within .venv
(.venv) $ python --version
Python 3.9.6
(.venv) $ exit
$

Package Management

Assume an update is desired to the existing app such that we need to add a new package dependency: Pendulum. Directly from the command line we can install and update the requirements files with a single command: `poetry add`.

$ poetry add pendulum
Using version ^2.1.2 for pendulum

Updating dependencies
Resolving dependencies... (0.7s)

Writing lock file

Package operations: 3 installs, 0 updates, 0 removals

  • Installing python-dateutil (2.8.1)
  • Installing pytzdata (2020.1)
  • Installing pendulum (2.1.2)

$ poetry show --tree pendulum
pendulum 2.1.2 Python datetimes made easy
├── python-dateutil >=2.6,<3.0
│   └── six >=1.5
└── pytzdata >=2020.1

Note: it’s possible to verify installed packages with `poetry show` to view similar output to `pipdeptree`.

With pendulum installed locally, update the existing main.py file to display the time in UTC.

# main.py
from fastapi import FastAPI
import pendulum 

app = FastAPI()

@app.get("/")
async def root():
    return {"message": f"The time is {pendulum.now('UTC')}"}

Start our app server with `poetry run` to validate the updates.

$ poetry run uvicorn main:app --reload
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [5255] using watchgod
INFO:     Started server process [5261]
INFO:     Waiting for application startup.
INFO:     Application startup complete.

...

$ http :8000
HTTP/1.1 200 OK
content-length: 58
content-type: application/json
date: Sat, 10 Jul 2021 18:21:22 GMT
server: uvicorn

{
    "message": "The time is 2021-07-10T18:21:22.768294+00:00"
}

(No More) Requirements

With Poetry managing the dependencies, we no longer need the pip-tools files anymore. We can safely `rm *requirements.*` files from the app directory; all of our requirements are defined in the pyproject.toml and poetry.lock files.

What if we need that requirements.txt file, though? Perhaps it’s used as part of a Docker build in CI/CD, or maybe it gets run by various terraform or sensible scripts during deploy. Regardless, `poetry export` will generate one from the current poetry.lock file.

$ poetry export --without-hashes
fastapi==0.66.0; python_version >= "3.6"
pendulum==2.1.2; (python_version >= "2.7" and python_full_version < "3.0.0") or (python_full_version >= "3.5.0")
pydantic==1.8.2; python_full_version >= "3.6.1" and python_version >= "3.6"
python-dateutil==2.8.1; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0"
pytzdata==2020.1; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0"
six==1.16.0; python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.5.0"
starlette==0.14.2; python_version >= "3.6"
typing-extensions==3.10.0.0; python_full_version >= "3.6.1" and python_version >= "3.6"

$ poetry export --dev -o dev-requirements.txt

The JBS Quick Launch Lab

Free Qualified Assessment

Quantify what it will take to implement your next big idea!

Our assessment session will deliver tangible timelines, costs, high-level requirements, and recommend architectures that will work best. Let JBS prove to you and your team why over 24 years of experience matters.

Get Your Assessment