# A First Look at Poetry > Learn how Poetry simplifies Python development by consolidating common tools for managing dependencies and environments efficiently - **Collection:** Blog post - **Published:** 2024-08-01 - **Author:** Anthony Campolo - **Canonical URL:** https://ajcwebdev.com/first-look-poetry/ - **Markdown URL:** https://ajcwebdev.com/first-look-poetry/index.md - **JSON URL:** https://ajcwebdev.com/first-look-poetry/index.json --- ## Overview Poetry is a comprehensive tool for managing Python dependencies and packaging. It is designed to handle project dependencies and package management by integrating several pieces of functionality you might otherwise seek from separate tools. ### Python Tools Replaced by Poetry Have you ever struggled to understand how and when to use tools like `virtualenv`, `venv`, `pipenv`, `pyenv`, `pip`, `PDM`, `conda`, and `miniconda`? If you have, I've got some good news. By integrating these separate concerns around Python development, Poetry significantly reduces the risk of descending into a nihilistic fit of rage, despair, and eventual insanity that typically accompanies the task of managing virtual environments in Python. Here’s a breakdown of how Poetry interacts with or replaces the need for other commonly used tools in Python development: - Poetry automatically creates/manages virtual environments eliminating the need for manually using `venv` or `virtualenv` to create isolated environments. - If you aren't working in a specific domain that benefits from Conda's features (like scientific computing), you may not need `conda` or `miniconda`. - There is no need to use `Pipenv` if you are using Poetry, as both serve similar purposes in dependency management and virtual environment creation. - Like `Pipenv`, `PDM` is another tool for dependency management and project setup that you wouldn't need if you're using Poetry. - If you work with multiple versions of Python, `pyenv` can still be useful alongside Poetry. - `pyenv` helps manage multiple Python versions on your system. - Poetry can be configured to use different Python versions managed by `pyenv` for different projects. - Poetry uses `pip` internally to install dependencies. - Understanding how `pip` works can be helpful, but you generally won't need to use `pip` directly when managing packages within a Poetry-managed project. To summarize, when using Poetry it can be beneficial to understand `pip` but not necessary for direct use. You may also find `pyenv` helpful if you need to switch between multiple Python versions. However, other tools like `venv`, `virtualenv`, `Pipenv`, `PDM`, and `conda` can be safely replaced by Poetry for most standard Python project workflows. If specific features of these tools are required for specialized tasks, consider that an opportunity to reevaluate whether those tasks are actually worth keeping around. ### What About Setuptools and Wheel `setuptools` and `wheel` are essential tools in the Python ecosystem, especially for packaging and distribution of Python projects. For most development work where you are using Poetry, you do not need to directly interact with `setuptools` or `wheel`, as Poetry abstracts much of the functionality of these tools. However, understanding how they work is beneficial, especially if you're dealing with Python packaging beyond the scope of typical application development, like creating libraries intended for public distribution. Basically, even if you are using Poetry, it's useful to understand what these tools do because they are foundational to how Python packages are built and distributed. `Setuptools` is a library designed to help you create and distribute Python packages. - It extends the `distutils` (distribution utilities) provided by the Python standard library. - It includes commands for building and installing packages, handling metadata like dependencies, and more. - Essentially, it provides the tools to package your Python project and manage its dependencies. - While Poetry handles project dependencies and can package projects on its own, `setuptools` remains a fundamental part of the Python packaging ecosystem that Poetry interacts with under the hood. - When you build a package with Poetry for distribution (e.g., creating a source distribution), it creates a `setup.py` dynamically if needed for compatibility, though this is abstracted away from the user. `Wheel` is a package format designed as a faster and more reliable method of installing Python software than re-building from source code each time. - A wheel is a pre-built distribution format that can speed up the installation process for Python packages. - Unlike a source distribution, which requires building and compiling at the time of installation, a wheel is a binary distribution that can be quickly installed. - Poetry can build wheel files directly when you prepare your project for distribution. - This means that when you publish a package using Poetry, it can create `.whl` files, which are then uploaded to package indices like PyPI. ## Getting Started This guide covers the essential aspects of Poetry for someone just learning the tool. It will walk you through the basics of using Poetry for your Python projects including installation, project setup, python version management, working with dependencies, using virtual environments, and configuring your projects and local machine. ### Install Poetry Install Poetry with the official install script: ```bash wrap=false curl -sSL https://install.python-poetry.org | python3 - ``` On MacOS, this will install `poetry` in `.local/bin` inside your home directory (`/Users/ajcwebdev/.local/bin` for example). If this directory is not already in your `PATH`, add the following to your `.zshrc` file: ```bash wrap=false export PATH="$HOME/.local/bin:$PATH" ``` Verify your installation completed successfully by checking your `poetry` version: ```bash wrap=false poetry --version ``` At the time of publication for this blog post, I was using Poetry version `1.8.3`. `poetry self` manages the Poetry installation itself with subcommands such as: - `add` for installing plugins - `show plugins` for listing installed plugins - `update` for updating Poetry Running the `update` command like so: ```bash wrap=false poetry self update ``` Gives the following output: ```txt wrap=false Updating Poetry version ... Using version ^1.8.3 for poetry Updating dependencies Resolving dependencies... (5.1s) No dependencies to install or update Writing lock file ``` ### Create and Build a New Project with Poetry To create a new project with Poetry, use the `poetry new` command: ```bash wrap=false poetry new ajcwebdev-poetry ``` This creates a directory structure for your project, including a `pyproject.toml` file, which is the heart of your Poetry project. It contains all the project configurations and orchestrates your project and its dependencies. Later in this post, we'll dive deeper into the various configuration files used by Poetry. For now, here's a basic example: ```toml # pyproject.toml [tool.poetry] name = "ajcwebdev-poetry" version = "0.1.0" description = "A getting started example repo for Poetry created by Anthony Campolo" authors = ["Anthony Campolo <12433465+ajcwebdev@users.noreply.github.com>"] readme = "README.md" [tool.poetry.dependencies] python = "^3.12" [build-system] requires = ["poetry-core"] build-backend = "poetry.core.masonry.api" ``` Under `tool.poetry.dependencies` in the `pyproject.toml` file, you can specify the Python version your project supports. In the above example, `python` is set to `^3.12` which allows any Python version greater than or equal to 3.12.0 and less than 4.0.0. You can show the project version by running: ```bash wrap=false poetry version # ajcwebdev-poetry 0.1.0 ``` For managing project versions, the project's version can be incremented with `version patch`: ```bash wrap=false poetry version patch # Bumping version from 0.1.0 to 0.1.1 ``` `poetry build` builds the source and wheels archives and can be used for specifying the output directory for build artifacts (`poetry build --output dist`). ```bash wrap=false poetry build ``` Output: ```txt wrap=false Creating virtualenv ajcwebdev-poetry-RnIQm9t6-py3.12 in /Users/ajc/Library/Caches/pypoetry/virtualenvs Building ajcwebdev-poetry (0.1.1) - Building sdist - Built ajcwebdev_poetry-0.1.1.tar.gz - Building wheel - Built ajcwebdev_poetry-0.1.1-py3-none-any.whl ``` ### Adding Installing and Updating Dependencies To add a dependency to your project, use the `poetry add` command. This will add the package to your `pyproject.toml` file and install it: ```bash wrap=false poetry add cowsay ``` Output: ```txt wrap=false Using version ^6.1 for cowsay Updating dependencies Resolving dependencies... (0.3s) Package operations: 1 install, 0 updates, 0 removals - Installing cowsay (6.1) Writing lock file ``` The command can be used for numerous types of dependencies including but not limited to: - Adding packages with specific version constraints (e.g., `poetry add pendulum@^2.0.5`) - Adding git dependencies (e.g., `poetry add git+https://github.com/sdispater/pendulum.git`) - Adding local directories or files as dependencies - Adding packages with extras (e.g., `poetry add "requests[security,socks]"`) To install all dependencies specified in your `pyproject.toml` file, use the `poetry install` command. ```bash wrap=false poetry install ``` This creates a `poetry.lock` file if it doesn't exist, ensuring consistent installations across different environments. `poetry export` exports the lock file to other formats, like `requirements.txt` which is important for compatibility with other tools or deployment processes. The `poetry install` command can also be used for: - Installing only specific dependency groups (e.g., `poetry install --only test,docs`) - Synchronizing the environment with the lock file (`poetry install --sync`) - Installing with or without compilation of Python bytecode (`poetry install --compile`) To update your dependencies to their latest versions (within the constraints specified in `pyproject.toml`), use the `poetry update` command to update the `poetry.lock` file with the new versions: ```bash wrap=false poetry update ``` This command can be used to update only specific packages (`poetry update requests toml`) or update dependencies without installing them (`poetry update --lock`). ## Managing Environments Poetry ensures each project works in an isolated environment so you can easily switch between different Python versions for your project. ### Creating and Activating Virtual Environments Poetry creates a virtual environment for your project. To run commands within this environment, use the `poetry run` command for executing scripts defined in `pyproject.toml` (e.g., `poetry run my-script`). Add the following to your `__init__.py` file in your project's root (`ajcwebdev_poetry` in my case): ```py # ajcwebdev_poetry/__init__.py import cowsay cowsay.cow("Hello, I'm a cow!") ``` Run this script: ```bash wrap=false poetry run python3 ajcwebdev_poetry/__init__.py ``` You did a cow. ``` _________________ | Hello, I'm a cow! | ================= \ \ ^__^ (oo)\_______ (__)\ )\/\ ||----w | || || ``` To activate the virtual environment in your shell, use the `poetry shell` command to spawn a shell within the project's virtual environment (useful for working directly within the project environment): ```bash wrap=false poetry shell ``` If you're a naive, noob Python developer like me, you might try immediately running `print("hello")` and get a weird error message related to `zsh`. This is because you haven't actually started a Python REPL yet. To explicitly use the Python interpreter to run Python code in the shell, use `python -c`: ```bash wrap=false python -c "print('hello')" # hello ``` Alternatively, you can start the actual Python interactive shell by running `python` and then executing your code: ```bash >>> print("hello") # hello ``` If you want to run Python commands directly in your shell without typing `python` each time, you might need to adjust your shell configuration. However, [Gibbity](https://openai.com/index/chatgpt/) (my good buddy and trusted scholar) told me it's generally considered good practice to use the Python interpreter explicitly to avoid confusion with shell commands. ### Environment Isolation and Default Python Version Poetry prioritizes project environment isolation as a core feature. This means Poetry always works isolated from your global Python installation. It first checks if it's running inside a virtual environment. - If yes, it uses that environment directly. - If no, it uses a previously created environment or creates a new one. By default, Poetry uses the Python version used during its installation to create virtual environments. - If this version is incompatible with the project's Python range, Poetry will try to find a compatible version. - If unable to find a compatible version, you'll be prompted to activate one explicitly. - If you use pyenv to manage Python versions, you can set `virtualenvs.prefer-active-python` to `true` which tells Poetry to use the current Python version of your shell. Example workflow: ```bash wrap=false pyenv install 3.9.8 pyenv local 3.9.8 # Activate Python 3.9 for the current project poetry install ``` ### Switching and Deleting Environments The `env` commands provide full control over environment management. Always ensure your project's `python` dependency in `pyproject.toml` matches the environment you're using. By mastering these environment management techniques, you can maintain clean, isolated, and version-specific environments for each of your Poetry projects. - Use the `env use` command to specify which Python version to use: - Full path: `poetry env use /full/path/to/python` - From PATH: `poetry env use python3.7` - Minor version: `poetry env use 3.7` - Revert to default: `poetry env use system` - Use `env remove` to delete environments: - By path: `poetry env remove /full/path/to/python` - By version: `poetry env remove python3.7` or `poetry env remove 3.7` - By name: `poetry env remove test-O3eWbxRl-py3.7` - Multiple at once: `poetry env remove python3.6 python3.7 python3.8` - All environments: `poetry env remove --all` - Removing the active environment will automatically deactivate it. ### Environment Information and Listing Environments - Display environment info: `poetry env info` - Show only environment path: `poetry env info --path` - Show only Python executable path: `poetry env info --executable` - List all virtual environments: `poetry env list` - Show full paths: `poetry env list --full-path` ## Poetry Configuration Files Poetry uses various configuration files that serve different purposes and are used in different contexts. These files include `pyproject.toml`, `poetry.toml`, and `config.toml`. - All three use the TOML format for configuration and all influence Poetry's behavior in some way. However, they differ in their scope and whether or not they should be publicly shared through version control. - By understanding the purpose and scope of each configuration file, you can effectively manage your Poetry projects and maintain consistency across different environments and team members. Here's a quick summary of when to use one or the other based on their scope and content plus whether they should be committed to version control or not: - Scope: - `pyproject.toml` is project-wide. - `poetry.toml` is project-specific but local. - `config.toml` is system-wide. - Version control: - `pyproject.toml` should be committed. - `poetry.toml` and `config.toml` typically should not be committed. - Content: - `pyproject.toml` contains project metadata and dependencies. - `poetry.toml` and `config.toml` contain Poetry-specific settings. `poetry config` allows editing Poetry configuration settings and repositories. Run `poetry config --list` to show current config variables. ### Global Configuration with config-toml The `config.toml` file contains global configuration settings for Poetry across all projects on your system. - It sets your personal defaults for Poetry across all projects and is created and modified using the `poetry config` command without the `--local` flag. - It's located in system-wide configuration directories that vary by OS. - Since it's globally scoped, it affects all projects on your system and sets default behaviors for Poetry. - It can be overridden by project-specific settings and is not shared between different machines or users. ### Project Specific Local Configuration with poetry-toml The `poetry.toml` file is used for local, project-specific Poetry configurations that shouldn't be shared with other developers. - It's created when you use the `poetry config` command with the `--local` flag and is located at the root of the project directory (alongside `pyproject.toml`). - It's scoped per-project but for local settings and overrides global Poetry settings for the specific project. - The file is useful for developer-specific settings that shouldn't affect other team members and typically should not be committed to version control. - Settings in `poetry.toml` override those in `config.toml` for a specific project. - To ensure you don't accidentally change global settings, always use `poetry config --local` to modify `poetry.toml`. ### Project Specific Version Controlled Configuration with pyproject-toml The `pyproject.toml` file is the main configuration file for a Poetry project and is essential for defining your project and its dependencies. - It's located in the root of project directories and is scoped per-project. - The file is meant to be version-controlled and shared among all developers working on the project. - Its contents include project metadata (name, version, description, authors, etc.), dependencies, and dev-dependencies. - It's used for build system requirements and tool-specific configurations (including Poetry-specific settings). `poetry check` validates the content of `pyproject.toml` and its consistency with `poetry.lock` which is important for ensuring project configuration is correct.