Chapter 23 — Virtual Environments and pip

Section 1: Why Projects Need Isolation

Every Python project depends on a set of external packages, and different projects often need different versions of the same package. If all packages are installed into one shared Python installation, those needs collide: upgrading a package for one project can break another project that depended on the older version, and installing a new package can introduce a dependency that conflicts with an existing one. The result is a tangled installation where it is hard to tell which project needs what, and where fixing one project risks breaking another. Isolation solves this problem by giving each project its own private space for packages, separate from the system Python and separate from other projects, so that changes in one project cannot affect another.

Isolation is not just about avoiding conflicts; it is also about clarity. When a project has its own environment, the list of packages it needs is visible and specific to that project, rather than mixed with packages needed by unrelated work. A new developer, or your future self returning to a project after months away, can see exactly what the project depends on, recreate that environment, and run the project with confidence that the packages match those used during development. Without isolation, that clarity is lost, because the shared installation accumulates packages from many projects over time, and nobody can tell which ones a particular project actually needs, or which versions it was tested with.

Isolation also protects the system Python itself. The Python installation that comes with an operating system, or that is used by system tools, may depend on specific package versions, and installing or upgrading packages globally can interfere with those tools, sometimes in ways that are difficult to repair. By keeping project packages in isolated environments, you leave the system Python untouched, using it only as a base from which to create environments, not as a place to install project dependencies. That separation is a professional habit: system tools stay stable, project dependencies stay contained, and the two never interfere with each other, even when a project needs a package version very different from the system's.

Finally, isolation makes projects portable and reproducible. A project with its own environment and a recorded list of dependencies can be set up on another computer by creating a fresh environment and installing the recorded packages, producing a setup that matches the original closely enough to run the project reliably. Without isolation and recording, portability depends on remembering which packages were installed globally, and in which versions, which is unreliable and often impossible after time has passed. Isolation, combined with a requirements record discussed later in this chapter, turns a project from something that works on one computer into something that can be set up anywhere, which is essential for collaboration, for deployment, and for your own future work on the project.

Check: How does installing all project packages into one shared Python installation create conflicts?

Section 2: What a Virtual Environment Is

A virtual environment is a self-contained directory that holds a copy or a link to a Python interpreter, along with its own folder for installed packages and its own scripts for running Python and its tools. When you activate a virtual environment, your terminal session is adjusted so that running python or pip refers to the environment's Python and its packages, rather than to the system Python. When you deactivate it, your session returns to normal, and the environment's packages are no longer visible. The environment itself is just a folder on disk, which you can create, use, delete, and recreate, and which does not affect other environments or the system installation, except through the activation mechanism that temporarily changes which Python is used.

It helps to think of a virtual environment as a workspace with its own shelf. The system Python is a shared library, and each project gets its own shelf where it keeps the packages it needs, without touching the shared library or other projects' shelves. When you work on a project, you take books from its shelf, meaning you run its Python with its packages. When you switch projects, you put those books back and take books from the other project's shelf, by deactivating one environment and activating another. The shelves themselves do not move or interfere; only the shelf you are currently using changes. That mental model captures both the isolation and the ease of switching that virtual environments provide, and it explains why they are the standard way to manage Python projects.

A virtual environment does not contain a complete, independent copy of everything Python needs. It typically links to the base Python installation for the standard library and the interpreter itself, while keeping its own package folder separate. That design makes environments lightweight to create, because they do not duplicate the entire Python installation, while still providing isolation for the packages that matter, which are the third-party packages you install for a project. The environment also has its own configuration that records which base Python it was created from, so that it uses the right interpreter version. Understanding that environments are lightweight links, not full copies, helps you see why creating one is quick and why deleting one is safe: you are removing a project's package folder and its activation scripts, not uninstalling Python itself.

Finally, a virtual environment is a tool for a project, not a project itself. It holds dependencies, but it does not hold your code, unless you choose to put your code inside its folder, which you generally should not. Your code lives in your project folder, and the environment lives alongside it, often in a subfolder with a conventional name, or in a separate location, depending on your preference and your tools. Keeping code and environment separate makes both easier to manage: code can be shared, versioned, and backed up without including the environment, which is large and machine-specific, and the environment can be recreated from a requirements list without affecting the code. That separation is a key part of professional project organisation, and it is discussed further in the next chapter on version control.

Check: What does activating a virtual environment change about your terminal session, and what does it leave unchanged?

Section 3: Creating Your First Environment

Creating a virtual environment is a simple command, run from your project folder, that sets up the environment's directory structure and its package folder. The standard library module for this purpose creates an environment in a folder you name, and that folder will contain subfolders for the interpreter, for packages, and for scripts. The folder name is a convention, not a rule, but common names make it clear that the folder is an environment and not project code, and they make it easy to exclude the folder from version control and backups, as discussed later. Choose a name that is obvious, use it consistently across projects, and keep the environment folder inside or near your project folder, so that its association with the project is clear.

Before creating an environment, decide which Python version it should use, because the environment is based on a particular Python installation. If you have only one Python installed, that decision is made for you, but if you have several, you should choose deliberately, based on the project's needs and on compatibility with the packages you plan to use. The creation command lets you specify which Python to base the environment on, and the environment will then use that Python's version. That version matters, because packages and language features can differ between Python versions, and a project developed with one version may not run identically with another. Recording the Python version alongside the project's requirements, even informally, helps future setup, because it tells the next person which Python to use when recreating the environment.

After creation, inspect the environment folder briefly, to understand its structure, even if you will not interact with it directly. You will see folders for binaries or scripts, for libraries, and for packages, and you may see a configuration file that records the base Python. You do not need to edit these files manually, and you generally should not, because the environment's tools manage them. Inspecting them once, however, demystifies the environment: it is not magic, but a folder with a predictable structure, created by a standard tool, that your terminal can activate. That understanding will help you later, when troubleshooting, because you will know where to look if activation fails or if a package seems to be missing.

Creating an environment does not install any third-party packages by default, except for the basic tools needed to install packages, which are included so that you can start adding dependencies immediately. That clean starting point is intentional: an environment begins empty, and you add only the packages your project needs, keeping its dependency list minimal and clear. Resist the urge to install many packages just in case, because each additional package adds a dependency that must be maintained, updated, and checked for conflicts. Start empty, add packages as your project genuinely needs them, and record each addition in your requirements list, so that the environment grows deliberately, reflecting the project's actual needs, rather than accumulating packages by habit.

Check: Why does a new virtual environment start with almost no third-party packages installed?

Section 4: Activating and Deactivating

Activating a virtual environment is the step that tells your terminal session to use that environment's Python and packages, instead of the system Python. Activation is usually done by running a script that lives inside the environment folder, and the exact script depends on your operating system and your shell, but the effect is the same: your session's path is adjusted so that the environment's scripts folder comes first, meaning that when you type python or pip, the environment's versions are found before the system's. Many shells also change the prompt to show the environment's name, giving you a visible reminder that an environment is active. That prompt change is a helpful signal, because working in the wrong environment, or in no environment, is a common source of confusion, where a package appears missing even though it is installed in a different environment.

Deactivating reverses activation, restoring your session's path and prompt to their normal state, so that python and pip again refer to the system versions. Deactivation is a simple command, available once an environment is active, and it does not delete the environment or its packages; it merely stops using them in the current session. You can activate a different environment afterwards, or work without an environment, as you choose. Deactivation is important when switching projects, because leaving one project's environment active while working on another can cause you to install packages into the wrong environment, or to run code with the wrong dependencies, producing confusing results. Make it a habit to deactivate when you finish working on a project, or to activate the correct environment when you start, and to check your prompt if you are unsure which environment, if any, is active.

Activation affects only the current terminal session, not other sessions, and not the system as a whole. That per-session behaviour is useful, because it lets you work on different projects in different windows, each with its own environment active, without interference. It also means that activation is not a permanent system change, and that forgetting to activate in a new session is a common, easily fixed mistake: simply activate, and the session will use the correct Python and packages. Understanding that activation is temporary and per-session prevents confusion when a package works in one window but not in another, because the difference is often just activation, not installation.

Finally, activation scripts are specific to shells and operating systems, so the command you use on one system may differ on another, and documentation or guides may show several variants. Do not be confused by those variants; they all achieve the same effect, adjusting the path for the current session, and the variant for your shell is the one to learn. If activation fails, check that the environment folder exists, that you are running the script from the correct location or with the correct path, and that you are using the script for your shell, because those are the most common causes of activation problems. Once activation works, verify it by checking which Python is being used, a step covered later, so that you know, with evidence, that you are working in the intended environment before installing packages or running code.

Check: Why does activation affect only the current terminal session, and why is that useful?

Section 5: Installing Packages with pip

pip is the standard tool for installing Python packages, and it works by downloading a package from a package index, which is a public repository of Python packages, and installing it into the currently active environment, or into the system Python if no environment is active. The basic command names the package to install, and pip handles the details: finding the package, downloading the appropriate version, installing its dependencies, and placing its files in the environment's package folder. That simplicity is deceptive, because behind it lies a system of version resolution, dependency management, and compatibility checking, but for everyday use, you only need to know the basic command and the importance of having the right environment active before you run it, so that packages go to the intended place.

Before installing, always check that the intended environment is active, because installing into the wrong environment is a common mistake that leads to confusing errors later, where code runs in one environment but a package was installed in another. Checking is easy: look at your shell prompt for the environment name, or check which Python and pip are being used, and confirm they point inside the environment folder. That check takes a second and prevents a class of problems that can take much longer to diagnose, because the package is actually installed, just not where the code is looking. Make the check a habit, especially after opening a new terminal, switching projects, or returning to a project after a break, when activation status is easiest to forget.

Installing a package also installs its dependencies, which are other packages it needs to work. pip resolves those dependencies automatically, choosing versions that are compatible with each other and with your Python version, where possible. That automatic resolution is convenient, but it can also introduce many packages you did not explicitly choose, and it can fail if dependencies conflict with packages already installed. When installation succeeds, note what was installed, including dependencies, because that information matters for recording requirements, discussed later. When it fails, read the error message carefully, as it often explains the conflict or the missing compatibility, and consider whether the package requires a different Python version, or whether an already installed package is blocking a compatible version. Installation failures are often dependency problems, not simple mistakes, and understanding that helps you respond appropriately.

Finally, install only packages you actually need, from sources you trust. The public package index contains many packages, of varying quality and maintenance, and installing a package adds code to your project that you are now responsible for, in the sense that its bugs, security issues, and updates affect your project. Choose well-known, well-maintained packages for important functionality, check their documentation and their maintenance status, and avoid installing packages on impulse, especially for trivial functionality you could implement simply yourself. Each package is a dependency, and dependencies are a commitment: they must be kept up to date, checked for conflicts, and understood well enough to debug when they interact with your code. Installing thoughtfully, in the right environment, is the foundation of healthy dependency management.

Check: Why should you check that the right environment is active before installing a package with pip?

Section 6: Specifying Versions

Specifying a version when installing a package means telling pip exactly which release you want, rather than accepting whatever is newest at the time. That precision matters for reproducibility: a project that was developed and tested with one version of a package may behave differently, or fail, with a newer version that changed its behaviour, removed a feature, or introduced a bug. By specifying versions, you record the versions that are known to work with your project, so that recreating the environment later produces the same setup, rather than a newer setup that has not been tested. Version specification is therefore not pedantry; it is how you freeze a working combination of packages, so that it can be recreated reliably.

Version specifiers can express different levels of strictness, from an exact version, which allows only that release, to a range, which allows any version within certain bounds, such as at least a minimum version but below a major version that might introduce breaking changes. Exact versions give the strongest reproducibility, but they can make updating harder, because every package must be updated explicitly. Ranges give more flexibility, allowing minor updates that fix bugs without changing behaviour, while still protecting against major changes that might break the project. Choosing between exact and ranged specifiers is a judgement about the project's needs: for a project that must be recreated exactly, such as a deployed application, exact versions are safer; for a project under active development, where updates are welcome, ranges may be more practical, provided they are reviewed when updating.

When you install without specifying a version, pip installs the newest compatible version available at that moment, which is convenient for trying a package, but which should not be left unspecified in a project's recorded requirements. After trying a package and confirming it works, record the version that was installed, so that future installations use that version, not a newer one that has not been tested. That recording step, moving from an exploratory installation to a recorded requirement, is an important discipline, because it converts a working setup on one computer into a reproducible setup anywhere. Without it, a project that works today may fail tomorrow, when a dependency updates, even though no project code changed, which is a confusing and frustrating failure.

Version conflicts arise when two packages require incompatible versions of a shared dependency, such as one requiring a version below a threshold and another requiring a version above it. pip will report such a conflict, and resolving it may involve updating one package, downgrading another, or finding alternative versions that are compatible. Conflicts are a normal part of dependency management, not a sign that you have done something wrong, and they are best resolved by understanding the requirements of each package, rather than by forcing an installation that ignores the conflict. Forcing can produce an environment where packages are installed but do not work together correctly, which is worse than a clear conflict message. Read conflict messages carefully, consider the trade-offs, and choose a combination that all packages support, even if it means using slightly older versions than the newest available.

Check: Why is recording the exact version that works better than leaving a requirement unspecified?

Section 7: Requirements Files

A requirements file is a plain text file that lists the packages a project needs, one per line, usually with version specifiers, so that the project's environment can be recreated by installing everything in the file with a single command. The conventional name for this file is requirements.txt, and it lives in the project folder, alongside the code, so that it is shared and versioned with the project. A requirements file turns a project's dependencies from a memory, or a series of installation commands that may be forgotten, into a document that anyone can read and use to set up the project. That document is essential for collaboration, for deployment, and for your own future work, because it answers the question, what does this project need, in a precise, runnable form.

Writing a requirements file starts with listing the packages you directly use in your code, with the versions you have tested. It is tempting to list every package installed in your environment, including dependencies of dependencies, but that approach makes the file harder to maintain, because dependencies change when direct packages update, and listing them explicitly can create conflicts. A cleaner approach is to list direct dependencies, which are the packages your code imports, and to let pip resolve their dependencies during installation, recording the full, resolved set separately if exact reproducibility is needed, through a freezing mechanism discussed later. That distinction, direct versus transitive dependencies, keeps the requirements file readable and focused on what the project actually uses, while still allowing exact recreation when needed.

Keeping a requirements file up to date is a habit, not a one-time task. Every time you install a new package for a project, add it to the requirements file, with its version, before you forget. Every time you remove a package from use, remove it from the file, so that future environments do not install unnecessary packages. Every time you update a package, update its version in the file, after testing that the project still works with the new version. That maintenance, done as part of each change, keeps the file accurate, while updating it only occasionally, from memory, risks omissions and errors. Treat the requirements file as part of the project, updated alongside code changes, not as documentation to be written at the end, when details have been forgotten.

Finally, a requirements file should be tested by recreating an environment from it, in a clean environment, and running the project's tests or main functionality, to confirm that the file is complete and correct. A file that is missing a package will cause a fresh environment to fail, even though the original environment works, because the original has the package installed but not recorded. That failure is valuable, because it reveals the omission before it affects a collaborator or a deployment, where it would be more embarrassing and harder to fix. Testing a requirements file by recreating an environment is therefore a verification step, similar to running tests, that should be done whenever the file changes significantly, and certainly before sharing a project. A requirements file that has been verified by recreation is trustworthy; one that has not is only a guess.

Check: What is the difference between direct dependencies and transitive dependencies in a requirements file?

Section 8: Freezing an Environment

Freezing an environment means recording the exact versions of all packages currently installed in it, including direct dependencies and their transitive dependencies, into a file that can be used to recreate that environment precisely. The pip tool provides a freeze command that outputs this list, with each package and its exact version, and that output is often saved to a requirements file, or to a separate lock file, depending on the project's conventions. Freezing is useful when you need exact reproducibility, such as for a deployed application, where the production environment must match the development environment as closely as possible, because even a minor version difference in a transitive dependency could change behaviour in a subtle way that tests did not catch.

Freezing differs from hand-writing a requirements file, in that freezing captures everything installed, while hand-writing captures only what you intend. That completeness is both a strength and a weakness. It is a strength for reproducibility, because it leaves nothing to resolution, ensuring that a recreated environment has exactly the same packages, in the same versions, as the original. It is a weakness for maintainability, because the frozen file includes many packages you did not directly choose, making it harder to read, and making updates more cumbersome, because updating one package may require updating many frozen entries to keep them compatible. For that reason, many projects maintain two records: a hand-written requirements file with direct dependencies, for readability and for development, and a frozen file with all dependencies, for exact deployment, updating the frozen file from the hand-written one through a controlled process.

When you freeze, review the output before saving it, to ensure it does not include packages that were installed experimentally and are no longer needed, or packages from a different project that leaked into the environment because the wrong environment was active. A frozen file that includes unnecessary packages will install those packages in every recreated environment, wasting time and adding potential conflicts, while a frozen file that includes packages from another project will confuse future readers about what the project actually needs. Cleaning the environment before freezing, by removing experimental packages, or freezing from a fresh environment created from the hand-written requirements, produces a cleaner frozen file that accurately reflects the project's needs.

Finally, freezing is a snapshot, not a substitute for testing. A recreated environment from a frozen file should still be tested, by running the project's tests or main functionality, to confirm that it works, because freezing records versions, but it does not guarantee that those versions work together in a new environment, especially if the Python version or the operating system differs. Freezing plus testing gives strong reproducibility: the same versions, verified to work in the recreated environment. Freezing alone, without testing, gives only a record, not a guarantee. Use freezing when exactness matters, keep the frozen file alongside the hand-written requirements, update it deliberately when dependencies change, and verify recreated environments by running the project, so that reproducibility is proven, not just assumed.

Check: Why might a project keep both a hand-written requirements file and a frozen file?

Section 9: Listing, Upgrading and Removing Packages

Listing installed packages is a basic environment management task, useful for checking what is installed, for verifying that an installation succeeded, and for reviewing an environment before freezing or sharing it. pip provides a list command that shows installed packages and their versions, in a readable table, and that list can be compared with a requirements file to spot discrepancies, such as a package installed but not recorded, or a package recorded but not installed. Listing is also useful when debugging, because a missing package, or a wrong version, is a common cause of import errors or unexpected behaviour, and listing quickly confirms whether the package is present and in which version. Make listing a routine check when an environment behaves unexpectedly, before assuming a code bug.

Upgrading a package means installing a newer version, usually to obtain bug fixes, new features, or security improvements. Upgrading should be done deliberately, not casually, because a newer version may change behaviour in ways that break your project, even if the change is minor. Before upgrading, note the current version, upgrade in a controlled way, and then run your project's tests to confirm that it still works with the new version. If tests fail, you can revert to the previous version, because you noted it, and investigate the change, rather than being stuck with a broken environment and no record of what changed. Upgrading one package at a time, rather than upgrading everything at once, makes it easier to identify which upgrade caused a problem, if one occurs, because the change is isolated to a single package.

Removing a package, or uninstalling it, is the reverse of installing, and it should also be done deliberately, with a corresponding update to the requirements file, so that the file remains accurate. Uninstalling a package that is still used by your code will cause import errors, so check that the package is truly no longer needed before removing it, by searching your code for its imports and by running tests after removal to confirm nothing breaks. Uninstalling a package that is a dependency of another package may also affect that other package, and pip will usually warn you about such dependencies, which you should read carefully before confirming removal. A clean environment, with only the packages a project needs, is easier to maintain, faster to recreate, and less prone to conflicts, so periodic removal of unused packages is good hygiene, provided it is done carefully and verified by tests.

Finally, keep a record of upgrades and removals, at least in the requirements file, and ideally in a brief note about why a change was made, especially for significant upgrades that involved behaviour changes or fixes. That record helps future developers, and your future self, understand why a particular version was chosen, or why a package was removed, without needing to rediscover the reasoning. Dependency management is an ongoing activity, not a one-time setup, and treating it as such, with listing, careful upgrading, careful removal, and accurate records, will keep your environments healthy and your projects reproducible over time, even as packages evolve and projects grow.

Check: Why should you upgrade one package at a time and run tests after each upgrade?

Section 10: Where Packages Live

Understanding where packages live helps you debug installation problems, because it lets you check whether a package is installed in the environment you think it is, and whether your code is looking in the right place. Packages installed by pip go into a site-packages folder inside the active environment, which is a subfolder of the environment's library folder. When Python runs, it searches for imports in a specific order, including the current directory, the standard library, and the site-packages folders of the active environment, among other places. That search order, called the module search path, determines which file is found when you import a name, and it explains why a package installed in one environment is not visible in another: the other environment's site-packages folder is not on the search path when that environment is not active.

You can inspect the search path from within Python, by examining the list of directories Python uses to find modules, which is a useful debugging step when an import fails despite a package appearing to be installed. If the environment's site-packages folder is not on the path, the environment may not be active, or Python may be running from a different installation than you expected. If the folder is on the path but the package is still not found, the package may not be installed in that folder, or it may be installed under a different name than the import name, which happens for some packages whose distribution name differs from their import name. Checking the site-packages folder's contents, by listing it, can confirm whether the package's files are actually present, and under what name, which resolves many import mysteries quickly.

Packages also include metadata, such as their version, their dependencies, and their location, which pip uses to manage them. That metadata lives alongside the package files in site-packages, in folders that pip can read to list, upgrade, or remove packages. Understanding that packages are files in a folder, plus metadata, demystifies pip: installing is copying files and metadata into site-packages, listing is reading metadata, upgrading is replacing files and metadata, and removing is deleting them. That file-based view helps you understand why deleting an environment folder removes all its packages, why copying a project folder does not copy its environment unless the environment folder is inside it, and why environments are machine-specific, because their files include paths and compiled components that may not work on another machine.

Finally, do not edit files in site-packages manually, except in rare debugging situations where you need to understand a package's behaviour, and even then, do so in a copy or with the understanding that your edits will be lost on upgrade and may break the package. Site-packages is managed by pip, and manual edits can cause pip's metadata to become inconsistent with the actual files, leading to confusing errors during later upgrades or removals. If you need to understand a package, read its files, but do not change them as a fix for your project; instead, fix your code to use the package correctly, or report a genuine package bug, or find an alternative package. Respecting site-packages as pip-managed territory keeps your environments clean and your dependency management reliable.

Check: Why is a package installed in one virtual environment not visible when another environment is active?

Section 11: Isolation in Action: Two Projects

Imagine you maintain two projects: one is a data tool that needs an older version of a data package, because its code was written against that version's behaviour, and the other is a new web tool that needs a newer version of the same package, because it uses a feature added in the newer release. Without virtual environments, you cannot satisfy both projects with a single installation, because only one version can be installed globally at a time. You would be forced to choose, breaking one project, or to constantly uninstall and reinstall versions as you switch projects, which is tedious and error-prone. With virtual environments, each project gets its own environment, with its own version, and both projects work correctly, side by side, without interference. That example, though simplified, captures the core value of isolation in everyday development.

Setting up the two environments follows the same pattern for each project: create an environment in or near the project folder, activate it, install the project's packages with the versions it needs, and record those versions in the project's requirements file. When you switch projects, you deactivate the first environment and activate the second, and your terminal session now uses the second project's packages. The switch takes seconds, and it is reliable, because the environments are independent folders that do not affect each other. You can even work on both projects in different terminal windows simultaneously, each with its own environment active, without any conflict, because activation is per-session, as discussed earlier. That workflow, once learned, becomes second nature, and it removes a whole class of version headaches from development.

Isolation also helps when you need to test a project with a different version of a package, without disturbing its working setup. You can create a second, temporary environment for the project, install the new version there, run the project's tests, and see whether they pass, all without touching the original environment. If the tests pass, you can update the original environment and its requirements file with confidence. If they fail, you can discard the temporary environment, and the original remains untouched and working. That ability to experiment safely, by creating disposable environments, is a powerful advantage of isolation, because it encourages trying updates and new packages without fear of breaking a working setup. Disposable environments are cheap to create and delete, so use them freely for experiments.

Finally, isolation clarifies responsibility when something goes wrong. If a project fails in its own environment, the problem is likely in that project's code, its recorded dependencies, or its environment setup, not in another project's packages, because other projects' packages are not visible. That narrowing saves debugging time, because you do not need to consider global installation state, which may include packages from many unrelated projects. If a project works in one environment but not in a freshly recreated one, the difference is likely in the requirements file, which may be incomplete, or in the Python version, which may differ, and those are specific, checkable causes. Isolation does not eliminate bugs, but it eliminates a confusing category of them, where projects interfere through shared packages, and that elimination is worth the small overhead of managing environments.

Check: How do separate environments let two projects use different versions of the same package?

Section 12: Conflicts and How to Resolve Them

Dependency conflicts occur when the packages a project needs cannot all be satisfied at once, because they require incompatible versions of a shared dependency, or because a required version is not available for your Python version or platform. pip reports conflicts with a message that names the conflicting packages and the versions they require, and reading that message carefully is the first step to resolving the conflict. The message may seem technical, but it essentially says: package A needs version X of a shared package, while package B needs version Y, and X and Y cannot both be installed. Understanding that statement, in terms of which packages are involved and what versions they need, turns a confusing error into a clear problem to solve, with a limited set of possible solutions.

Possible solutions include updating or downgrading one of the conflicting packages to a version whose requirements are compatible with the others. For example, if package A requires an older version of a shared dependency, and package B requires a newer version, perhaps a newer version of A exists that supports the newer shared version, or an older version of B exists that supports the older shared version. Finding such a compatible combination may involve checking the packages' documentation or release notes, which often state which versions of dependencies they support, and trying a combination that satisfies all requirements. That trial-and-error, guided by the conflict message, is a normal part of dependency management, and it is best done in a disposable environment, so that failed attempts do not disturb a working setup.

Another solution is to reconsider whether both conflicting packages are truly needed. Sometimes a project includes a package that is no longer used, or that was installed experimentally, and removing it eliminates the conflict entirely. Sometimes two packages provide overlapping functionality, and choosing one over the other simplifies the dependency set and removes the conflict. Sometimes a package can be replaced with a simpler alternative, or with code you write yourself, if its functionality is small. Those reconsiderations are worth making, because a smaller dependency set is easier to maintain and less prone to conflicts, and a conflict is often a signal that a project's dependencies have grown more complex than necessary. Simplifying dependencies is not just a fix for a conflict; it is an improvement to the project.

If no compatible combination can be found, and both packages are essential, you may need to accept a compromise, such as using an older version of the project itself, waiting for an updated package that resolves the conflict, or restructuring the project to separate the conflicting parts into different environments or services, though that last option is beyond the scope of a small project. In such cases, document the conflict and the decision, so that future work can revisit it when packages update. Conflicts are not failures of the developer; they are a normal consequence of a large, evolving ecosystem of packages, where different projects evolve at different speeds. Resolving them patiently, by reading messages, trying compatible combinations, and simplifying where possible, is a professional skill that improves with practice and with familiarity with the packages you use most.

Check: What are three possible ways to resolve a dependency conflict between two packages?

Section 13: Reproducibility and Sharing a Project

Reproducibility means that another person, or you on another computer, can recreate a project's environment and run it successfully, using only the project's files and documentation. Achieving reproducibility requires three things: a recorded list of dependencies, with versions, a recorded Python version, and instructions for setting up the environment, including how to create it, activate it, and install dependencies. Without any one of those three, recreation may fail or produce a different setup that behaves differently. With all three, recreation is a straightforward sequence of steps that can be followed reliably, even by someone unfamiliar with the project, which is essential for collaboration, for deployment, and for long-term maintenance, where the original developer may no longer be available.

Instructions for setup should be brief, clear, and tested, ideally by following them yourself in a fresh environment, to confirm they work. A typical setup instruction set includes: create a virtual environment, activate it, install dependencies from the requirements file, and run the project's tests or main entry point to verify the setup. Each step should be stated in terms that work on the target system, noting any differences between operating systems where relevant, such as activation scripts. Instructions that assume knowledge, such as activate the environment without saying how, or that omit a step, such as installing dependencies, will frustrate new users and lead to support requests that could have been avoided by a complete, tested set of instructions. Treat setup instructions as part of the project, to be maintained alongside code and requirements.

Sharing a project also means deciding what not to share. The environment folder itself should not be shared, because it is large, machine-specific, and contains paths and compiled files that may not work on another machine. Instead, share the code, the requirements file, and the setup instructions, and let the recipient create their own environment from those. That approach is cleaner, smaller, and more reliable, because the recipient's environment will be built for their machine and Python version, rather than copied from yours. The same principle applies to version control, discussed in the next chapter: environment folders are excluded from version control, while requirements files are included, because requirements are portable and environments are not. Sharing the recipe, not the built environment, is the professional practice.

Finally, reproducibility should be verified periodically, not just assumed. As a project evolves, its dependencies change, its Python version may update, and its setup instructions may become outdated, if they are not maintained alongside code changes. Periodically recreating the environment from scratch, following the instructions exactly, and running the project's tests, confirms that reproducibility still holds, and reveals omissions before they affect a collaborator. That verification, done occasionally, is a small investment that prevents the embarrassing situation where a project works on the developer's machine but cannot be set up anywhere else, which is a common failure in projects that neglect reproducibility. A project that can be recreated reliably is a project that can be shared, deployed, and maintained, which is the goal of professional development.

Check: What three things must a project provide for another person to recreate its environment reliably?

Section 14: Excluding Environments from Version Control

Version control, covered in detail in the next chapter, tracks changes to a project's files over time, but not every file in a project folder should be tracked. Virtual environment folders are a prime example of files that should not be tracked, because they are large, containing thousands of files for installed packages, machine-specific, containing paths and compiled components tied to one computer, and recreatable, because they can be rebuilt from a requirements file. Tracking an environment folder would bloat the project's history, slow down version control operations, and cause conflicts when different developers have different environments, with different paths or package builds. Excluding the environment folder, while tracking the requirements file, gives the best of both worlds: the project's dependencies are recorded, in portable form, while the bulky, machine-specific environment itself is left out.

Exclusion is typically achieved with an ignore file, which lists patterns for files and folders that version control should ignore, such as the environment folder's name. Once ignored, the environment folder will not be added to version control accidentally, and version control commands will not show it as an untracked file cluttering the status output. Setting up that ignore file early, when a project is created, prevents accidental inclusion, which is harder to undo cleanly once an environment has been committed, because removing it from history requires rewriting history, which is disruptive in a shared project. Early exclusion is therefore a habit to adopt at project creation, alongside creating the environment and writing the requirements file, as part of a standard project setup checklist.

The same exclusion principle applies to other generated or machine-specific files, such as compiled bytecode caches, editor settings that are personal, and operating system files that appear in project folders. Those files, like environments, are recreatable or personal, and tracking them adds noise without value. A well-maintained ignore file lists all such patterns for a project, keeping the version-controlled files focused on source code, documentation, configuration that is shared, and requirements, which are the files that define the project and that collaborators need. Reviewing the ignore file periodically, as a project grows and new tools are used, ensures it stays accurate, and prevents new kinds of generated files from being tracked accidentally.

Finally, excluding an environment from version control does not mean losing it. The environment remains on your computer, usable for development, and it can be recreated at any time from the requirements file, as discussed earlier. Exclusion is about what is shared, not about what is kept locally. A new collaborator who obtains the project will create their own environment, following the setup instructions, and will have their own environment folder, ignored in their copy as well. That workflow, each developer with their own ignored environment, built from shared requirements, is the standard professional practice, and it keeps version control clean, fast, and focused on the files that truly define the project. Learn it early, and apply it to every project you start.

Check: Why should a virtual environment folder be excluded from version control, while its requirements file is included?

Section 15: System Python Versus Project Python

The system Python is the Python installation that comes with an operating system or that is installed for general use, and it may be used by system tools or by other applications. Project Python, in the context of virtual environments, is the Python inside a project's environment, which is based on a system Python but has its own package folder. Distinguishing the two is important, because installing packages into the system Python affects everything that uses it, including system tools, while installing into a project environment affects only that project. Beginners sometimes install packages globally, into the system Python, because it seems simpler, not realising that they are modifying a shared installation that other tools depend on, and that they are creating conflicts for future projects. Using project environments avoids that risk, by keeping project packages separate from the system.

System Python should generally be left alone, except for installing tools that are meant to be available globally, such as the tool used to create virtual environments itself, which is often installed globally so that it can be used for any project. Even then, care should be taken, because global installations can still conflict with system tools, and some operating systems discourage or restrict modifying the system Python for that reason. A safer approach, where available, is to use a user-level installation for global tools, or to use a dedicated tool manager, but the details depend on the system, and the key principle remains: keep the system Python as clean as possible, and put project dependencies in project environments. That principle protects both system tools and projects, by preventing interference in either direction.

Project Python, by contrast, is disposable and project-specific. If a project environment becomes corrupted, or if its packages become tangled through experimental installations, you can delete it and recreate it from the requirements file, without affecting the system Python or other projects. That disposability encourages experimentation, because the cost of a mistake is low: a broken environment can be rebuilt in minutes, while a broken system Python can be difficult and time-consuming to repair. It also encourages cleanliness, because an environment that has accumulated experimental packages can be discarded and rebuilt cleanly, rather than painstakingly cleaned package by package. Think of project environments as temporary workspaces, not as permanent installations, and you will use them more freely and more safely.

When you run Python code, be aware of which Python you are using: system or project. That awareness comes from checking your environment activation, and from checking which Python is found when you run it, as discussed earlier. A program that runs with the system Python but expects project packages will fail with import errors, not because the packages are missing, but because they are in a project environment that is not active. A program that runs with a project Python but expects system packages may also fail, if those packages were never installed in the project environment. Both failures are environment mismatches, not code bugs, and recognising them as such saves debugging time. Always know which Python you are using, and ensure it matches the project's environment, before investigating a code problem.

Check: Why is installing project packages into the system Python risky for both the system and future projects?

Section 16: Recreating an Environment from Scratch

Recreating an environment from scratch is a valuable skill, both for recovering from a broken environment and for verifying that a project's requirements are complete. The process is straightforward: delete the old environment folder, or move it aside if you want to keep it as a backup, create a new environment in its place, activate it, and install dependencies from the requirements file. After installation, run the project's tests or main functionality to confirm that the recreated environment works. That sequence, delete, create, activate, install, verify, should become familiar, because you will use it whenever an environment becomes corrupted, when you want a clean setup for testing, or when you set up a project on a new computer. Practising it on a small project, where the stakes are low, builds confidence for using it on larger projects later.

Recreation also serves as a test of the requirements file, as discussed earlier. If the recreated environment fails because a package is missing, the requirements file is incomplete, and it should be updated to include the missing package, with its version, before the project is shared. If it fails because a version is incompatible, the requirements file may need a version adjustment, or the code may need updating to work with a newer version. Those failures, discovered during recreation, are valuable, because they reveal gaps that would otherwise affect collaborators. Recreation is therefore not just a recovery technique; it is a verification technique, and it should be done periodically, especially after significant dependency changes, to ensure the project's recorded dependencies remain accurate and complete.

When recreating, be careful to distinguish between deleting an environment and deleting project code. The environment folder is separate from the code folder, or is a clearly named subfolder within the project, and deleting it should not affect code, documentation, or data, provided you have set up the project correctly. Before deleting, confirm the folder you are deleting is indeed the environment, by checking its name and its contents, and by ensuring that project code lives elsewhere or is backed up. That caution prevents the distressing mistake of deleting code while intending to delete an environment, which, while recoverable from version control if the code is tracked, is still disruptive. A moment's checking before deletion is a worthwhile habit, especially when using commands that delete folders permanently.

Finally, recreation is an opportunity to clean up. An old environment may contain packages installed experimentally, versions that are no longer needed, or configurations that have become tangled. A fresh environment, built only from the current requirements file, is clean by definition, containing only what the project currently needs, in the versions currently recorded. That cleanliness improves performance, reduces conflicts, and makes the environment easier to understand. Do not hesitate to recreate an environment when it feels messy or when you are unsure what is installed in it; recreation is cheap, fast, and safe, provided the requirements file is accurate, and it is often faster than trying to clean an environment package by package. A willingness to recreate, rather than to nurse a messy environment, is a sign of mature environment management.

Check: How does recreating an environment from scratch test whether a requirements file is complete?

Section 17: Troubleshooting Activation Problems

Activation problems are common for beginners, and they usually have simple causes that can be diagnosed systematically. The most common cause is running the wrong activation script for your shell or operating system, because environments contain different scripts for different shells, and using the wrong one may fail silently or produce an error. Check the environment's scripts folder, identify the script for your shell, and use that one, following the documentation for your system if you are unsure. Another common cause is running the script from the wrong location, or with an incorrect path, such as a relative path that does not point to the environment folder from your current directory. Using an absolute path, or changing to the project folder first, often resolves path problems, because it removes ambiguity about where the environment lives.

Another activation problem is that activation appears to succeed, but the prompt does not change, or Python still refers to the system installation. That can happen if the shell's configuration overrides the prompt change, or if the activation script was run in a way that does not affect the current session, such as running it as a separate command rather than sourcing it, depending on the shell. The distinction between running a script and sourcing it matters for activation, because activation works by changing the current session's environment, which a separately run script cannot do, as it runs in its own process that exits without affecting the parent session. Sourcing, which runs the script in the current session, is required for activation, and using the wrong method is a common mistake. If activation seems to have no effect, check whether you sourced the script correctly for your shell, and whether your shell's documentation describes a specific activation command.

Permissions can also cause activation problems, especially if the environment folder was created with different permissions, or if scripts are not marked as executable where required. Check the folder's permissions, and ensure you have read and execute access to the activation script and to the environment's folders. On some systems, security settings may block running scripts from certain locations, requiring an adjustment or a different approach, but such cases are less common and usually produce a clear security message. When in doubt, recreate the environment, which often resolves permission problems, because a fresh environment will have correct permissions for the current user, provided it is created in a location you own, such as your project folder or home directory, rather than in a system location that requires special permissions.

Finally, if activation continues to fail, verify the environment itself by checking its folder structure: does it contain the expected subfolders, the interpreter, and the activation scripts? If the folder is incomplete or corrupted, perhaps because creation was interrupted, recreating it is the simplest fix. Document the working activation command for your system, once you find it, in your project's setup instructions, so that you and collaborators can use it reliably in future. Activation problems, while frustrating at first, become rare once you know the correct command for your shell and understand that activation is per-session and requires sourcing, and troubleshooting them systematically, by checking script, path, method, and permissions, will resolve most cases quickly.

Check: Why must an activation script be sourced in the current session, rather than run as a separate process?

Section 18: Choosing Packages Wisely

Choosing packages wisely is an important part of dependency management, because each package you add becomes a part of your project that you depend on, even though you did not write it. A well-chosen package saves time, provides tested functionality, and is maintained by a community that fixes bugs and security issues. A poorly chosen package can introduce bugs, security vulnerabilities, or maintenance burdens, and can make your project harder to update, because it may be abandoned, poorly documented, or incompatible with other packages you need. Choosing wisely therefore involves evaluating a package before installing it, rather than installing the first package that appears to solve a problem, without checking its quality or suitability.

Evaluation criteria include maintenance: is the package actively maintained, with recent updates and responsive maintainers, or has it been abandoned, with no updates for a long time and unanswered issues? Documentation: does it have clear documentation, with examples and explanations, or is it poorly documented, requiring you to guess how to use it? Popularity and community: is it widely used, with many users who would notice and report problems, or is it obscure, with few users and little scrutiny? Compatibility: does it support your Python version and your other dependencies, or does it require versions that conflict with your project? Licence: is its licence compatible with your project's intended use, a topic covered in a later chapter on licences and open source? Those criteria, checked briefly before installation, help you avoid packages that will cause problems later, even if they seem convenient now.

Also consider whether you need a package at all. For small, simple functionality, writing a few lines of code yourself may be better than adding a dependency, because your own code is simpler to understand, easier to maintain, and has no external dependencies that could break or become abandoned. The standard library, which is included with Python, already provides a great deal of functionality, and many tasks can be accomplished with it alone, without external packages. Before installing a package, check whether the standard library already offers what you need, and whether the package's benefit outweighs its cost in terms of dependency management. That cost-benefit thinking, applied to each package, keeps dependency sets lean and projects maintainable, by avoiding unnecessary dependencies that add risk without proportional benefit.

Finally, once you choose a package, learn to use it well, by reading its documentation and understanding its main concepts, rather than copying examples blindly. A package used without understanding can produce subtle bugs, especially when its behaviour in edge cases differs from your assumptions, and debugging those bugs requires understanding the package, not just your code. Keep notes on packages you use frequently, including their purpose, their main functions, and any quirks you discovered, so that future use is faster and more reliable. Choosing wisely, using thoughtfully, and maintaining dependencies carefully are habits that distinguish a professional developer from one who accumulates packages indiscriminately, and they will serve you well as your projects grow in size and importance.

Check: What criteria should you check before adding a new package as a dependency?

Section 19: Dependency Safety Basics

Dependency safety means being aware that packages you install run code on your computer, and that malicious or compromised packages could harm your system or steal data, even though most packages are legitimate and useful. The public package index is an open repository, where anyone can publish packages, and while it has safeguards, it cannot guarantee that every package is safe, any more than an app store can guarantee every app. That reality calls for caution, not paranoia: use well-known packages from trusted authors, check a package's reputation before installing it, and be wary of packages with names very similar to popular packages, which may be attempts to trick users into installing the wrong package, a practice sometimes called typosquatting. A moment's checking before installation can prevent a serious security problem.

Safety also involves keeping dependencies up to date, because older versions may contain known security vulnerabilities that have been fixed in newer versions. Regularly reviewing your dependencies, checking for updates, and applying security updates promptly, while testing that your project still works after updating, is a basic safety habit. That habit requires balance, because updating can also introduce breaking changes, as discussed earlier, so updates should be done deliberately, one at a time, with tests, rather than blindly upgrading everything at once. Security updates, however, should be prioritised, because the risk of a known vulnerability being exploited often outweighs the risk of a breaking change, which tests can catch. Document security updates, so that you know when they were applied and why.

Another safety practice is to install packages only in virtual environments, never globally, except for trusted tools, because a malicious package installed globally could affect system tools and other projects, while one installed in a project environment is contained to that project, though it can still harm that project's data and, if run with sufficient permissions, the user's files. Virtual environments are isolation, not security sandboxes: they do not prevent a package from accessing your files or network, because Python code runs with your user's permissions regardless of environment. Understanding that limitation is important, because it prevents a false sense of security: an environment isolates dependencies from each other, but it does not isolate code from your system. Only install packages you trust, from authors and sources you have reason to trust, and be especially careful with packages that are new, obscure, or that request unusual permissions or network access.

Finally, review what a package does, at least at a high level, before relying on it for sensitive tasks, such as handling passwords, financial data, or personal information. For such tasks, prefer well-established packages with a strong security track record, and avoid obscure packages whose code you have not reviewed and whose maintainers are unknown. That caution is not about distrusting the open-source community, which produces excellent, secure software, but about recognising that trust should be earned through reputation, maintenance, documentation, and community scrutiny, not assumed for every package. Dependency safety, like other safety practices in this book, is defensive and proportionate: choose trusted packages, keep them updated, install them in environments, and review them before use for sensitive tasks, and you will greatly reduce the risk while still benefiting from the rich ecosystem of Python packages.

Check: Why does a virtual environment not make an untrusted package safe to install and run?

Section 20: A Clean Environment Workflow

A clean environment workflow is a repeatable sequence of steps for starting, working in, and finishing with a project environment, and adopting such a workflow prevents many common environment mistakes. The workflow begins when you start a project: create a project folder, create a virtual environment for it, activate the environment, and create an empty requirements file, ready to record dependencies as you add them. That initial setup, done once per project, establishes the isolation and recording habits from the start, rather than trying to retrofit them later, when dependencies have already become tangled. Starting clean is easier than cleaning up later, and it sets a professional tone for the project, signalling that dependencies will be managed deliberately.

During development, the workflow continues with small, consistent steps: activate the environment when you start work, check that it is active before installing or running code, install packages only when needed, record each installation in the requirements file immediately, and run tests after installing or updating a package, to confirm the project still works. Those steps, repeated for each change, keep the environment and the requirements file in sync, so that they never drift apart, which is the most common cause of recreation failures. Drifting occurs when packages are installed but not recorded, or recorded but not installed, and it is prevented by the habit of recording immediately, not later, when details are forgotten. A workflow that makes recording immediate, such as updating the requirements file as part of the installation step, is more reliable than one that relies on memory.

When you finish work for the day, or switch projects, deactivate the environment, so that the next session starts cleanly, without leftover activation from a previous project. That deactivation step, though simple, prevents the confusion of working in the wrong environment, which is a frequent beginner mistake. When you return to a project, reactivate its environment before doing anything else, and verify activation by checking the prompt or the Python path, so that you know you are in the right environment before you install, run, or test. Those bookend steps, activate at start and deactivate at finish, frame each work session clearly, and they become automatic with practice, like saving a file or committing a change in version control.

Periodically, as part of the workflow, recreate the environment from the requirements file, to verify reproducibility, and clean up unused packages, to keep the environment lean. That periodic verification, perhaps when you finish a feature or before sharing a project, catches drift and incompleteness early, when they are easy to fix. A clean environment workflow is therefore not a rigid set of rules, but a rhythm: create, activate, install and record, test, deactivate, and periodically recreate to verify. That rhythm, practised consistently, will make virtual environments feel natural rather than burdensome, and it will give you confidence that your projects are isolated, reproducible, and ready to share, which is the goal of this chapter.

Check: What are the bookend steps of a clean environment workflow at the start and end of a work session?

Section 21: Transitive Dependencies Explained

Transitive dependencies are packages that your project needs indirectly, because a package you directly use depends on them, even though your code never imports them directly. For example, if you install a web framework, that framework may depend on a templating package, a routing package, and a utilities package, all of which pip installs automatically, even though you never asked for them by name. Those indirect packages are transitive dependencies, and they are a normal part of modern software, because packages are built by reusing other packages, rather than reinventing every function. Understanding transitivity helps you interpret pip's output, which lists many packages after a single installation, and helps you understand why a conflict can arise from a package you never chose directly.

Transitive dependencies matter for maintenance, because they can change when you update a direct dependency, even if you did not intend to change them. Updating a framework to a newer version may require newer versions of its transitive dependencies, which pip will install as part of the update, potentially changing behaviour in parts of your project that use those transitive packages indirectly, through the framework. That indirect change is a reason to test thoroughly after updates, even when the update seems minor, because the visible change, a framework version bump, may hide invisible changes in its dependencies. It is also a reason to freeze environments for deployment, as discussed earlier, because freezing captures transitive versions exactly, preventing unexpected changes during recreation.

Transitive dependencies also matter for security and for understanding your project's true footprint. A project that directly uses five packages may actually depend on fifty packages in total, once transitivity is counted, and each of those fifty is code that runs as part of your project, with its own bugs and vulnerabilities. That footprint is not a reason to avoid dependencies, because reuse is valuable, but it is a reason to be aware of it, and to choose direct dependencies whose transitive footprint is reasonable and well-maintained. A direct package that pulls in dozens of transitive packages for a small feature may be a poor choice compared with a simpler package, or with standard library code, that achieves the same result with a smaller footprint. Evaluating a package's transitive dependencies, by looking at its listed requirements before installing, is a useful part of choosing wisely.

Finally, do not try to manage transitive dependencies manually, by pinning every transitive package by hand, unless you have a specific reason, such as a deployment requirement for exact reproducibility, in which case freezing, not hand-pinning, is the appropriate tool. Manual management of transitive dependencies is error-prone, because transitive requirements change when direct packages update, and hand-pinned versions can quickly become incompatible. Let pip resolve transitive dependencies from your direct requirements, freeze the result when exactness is needed, and focus your manual attention on direct dependencies, which are the packages you actually chose and use. That division of labour, you manage direct dependencies, pip manages transitive ones, with freezing as a snapshot when needed, is the standard, sustainable approach to dependency management in Python projects.

Check: What is a transitive dependency, and why can updating a direct package change transitive packages too?

Section 22: Version Specifiers in Detail

Version specifiers are expressions that tell pip which versions of a package are acceptable, and learning their basic forms helps you write requirements files that are both flexible and safe. An exact specifier allows only one version, giving maximum reproducibility but requiring explicit updates. A minimum specifier allows that version and any newer version, giving maximum flexibility but risking breaking changes from future major releases. A range specifier allows versions between a minimum and a maximum, often excluding the next major version, which is a common compromise: it allows bug fixes and minor improvements, while protecting against major changes that are more likely to break compatibility. That compromise, allowing minor updates but not major ones, is widely used because it balances freshness with stability, and it is a good default for many projects, unless exact reproducibility is required.

Compatible release specifiers express a similar idea more concisely: they allow updates that are compatible with a given version, typically meaning updates that do not change the leftmost version component that is specified, though the exact rule depends on the specifier's form. You do not need to memorise every specifier syntax to use pip effectively, but you should understand the concept: a specifier is a rule about which versions are acceptable, and different rules trade off between allowing updates and preventing breaking changes. When you write a requirements file, choose specifiers deliberately, based on how much change you are willing to accept without testing, and document any unusual choices, so that future readers understand why a particular range was chosen. A specifier chosen thoughtlessly, such as allowing any future version, may cause a future installation to use a version that breaks the project, even though no project code changed.

Version numbering itself follows conventions that help you interpret specifiers, though not all packages follow them strictly. Many packages use a three-part version, with major, minor, and patch components, where major changes may break compatibility, minor changes add features in a compatible way, and patch changes fix bugs without adding features. That convention, when followed, lets you reason about updates: a patch update is usually safe, a minor update is usually safe but may add features, and a major update requires careful testing. Always test after updating, regardless of how minor the version change appears, because the convention is a promise that is sometimes broken, intentionally or accidentally.

Finally, be consistent in how you specify versions across a project's requirements file. Mixing exact versions for some packages, ranges for others, and unspecified versions for the rest makes the file harder to understand and harder to maintain, because the reader cannot tell whether the inconsistency is deliberate or accidental. Choose a policy, such as exact versions for deployment and compatible ranges for development, or exact versions throughout, and apply it consistently, with exceptions documented where they occur. Consistency makes requirements files readable, makes updates predictable, and makes it easier to spot a version that was left unspecified by mistake. Version specifiers are a small part of a project, but they have an outsized effect on reproducibility and maintenance, so treat them with the care they deserve, and review them when dependencies change.

Check: How does a range specifier balance allowing updates with protecting against breaking changes?

Section 23: The Standard Library Versus External Packages

Python's standard library is a large collection of modules included with every Python installation, covering files, data formats, networking, testing, and many other common tasks, without needing any installation. External packages, installed with pip, extend Python beyond the standard library, providing specialised functionality, newer approaches, or more convenient interfaces for tasks the standard library handles only basically. Choosing between the standard library and an external package is a recurring decision in Python development, and it should be based on need, not on habit. If the standard library already provides what you need, clearly and reliably, using it avoids adding a dependency, with all the maintenance that entails. If it does not, or if its approach is cumbersome for your task, an external package may be justified, provided it is well-chosen, as discussed earlier.

The standard library's strength is stability and availability. Because it is included with Python, code that uses only the standard library can run anywhere Python runs, without installing anything, which makes it highly portable and reproducible. Its modules are also stable, with long-term support and careful attention to backward compatibility, so code written against the standard library is less likely to break due to a dependency update. That stability is why this book's capstone project, described in the outline, uses only the standard library: it ensures the project can be set up and run without external dependencies, making it accessible to learners and robust over time. For many tasks, especially learning projects, small tools, and scripts, the standard library is sufficient, and using it teaches you Python itself, rather than a particular package's interface.

External packages' strength is specialisation and convenience. A package may provide a simpler interface for a complex task, such as web development, data analysis, or image processing, where the standard library's tools are low-level or incomplete. A package may also provide performance improvements, or features that the standard library does not offer, or a community and ecosystem around a particular domain, with examples, tutorials, and support. Those benefits can justify a dependency, especially for larger projects where the time saved by using a package outweighs the cost of maintaining it. The key is to weigh benefit against cost for each task, rather than defaulting to external packages for everything, or refusing them on principle. A balanced approach, standard library by default, external packages where they add clear value, is a good rule for most projects.

Finally, learning the standard library well makes you a better judge of external packages, because you can compare a package's offering with what Python already provides, and you can recognise when a package is merely a thin wrapper around standard functionality, adding little value beyond convenience. It also makes you more versatile, because standard library skills transfer to any Python environment, while package-specific skills transfer only where that package is available. Invest time in the standard library, especially its modules for files, data, testing, and utilities, which are covered in this book, and treat external packages as additions to that foundation, not as replacements for it. That foundation-first approach will serve you well, whether you later specialise in web development, data analysis, or another domain where external packages are central.

Check: When is using the standard library preferable to adding an external package, and when is a package justified?

Section 24: Multiple Python Versions

Managing multiple Python versions is a related challenge to managing multiple package sets, because different projects may need different Python versions, or a single computer may have several Pythons installed for different purposes. A virtual environment is tied to the Python version it was created from, so an environment created with one Python version will use that version, even if other versions are installed. That tie is useful, because it ensures a project's Python version is consistent, but it also means that creating an environment requires choosing the right base Python, as discussed earlier. If you create an environment with the wrong Python version, perhaps because you did not specify which Python to use and the default was not the one you intended, the environment will use that wrong version, and packages or language features that depend on a specific version may fail or behave differently.

To manage multiple Pythons, you need to know which Pythons are installed on your system and how to refer to each one. Different systems handle this differently: some use versioned commands, some use a launcher that lets you specify a version, and some rely on paths. The details are system-specific, and you should learn the method for your system, rather than assuming a universal command. What matters conceptually is that you can identify each Python installation, by its version and its location, and that you can create an environment from a specific one, rather than always using the default. That control is essential when a project requires a particular Python version, because the default may change over time as new Pythons are installed, while the project's requirement stays fixed.

Version managers, which are tools that install and switch between multiple Python versions, can help, by providing a unified way to install Pythons and to select one for a project or a session. Whether you use a version manager or manage Pythons manually, the principles are the same: know which Python each project needs, create its environment from that Python, record the version in the project's documentation, and verify the version when you activate the environment, by checking which Python is in use. Those habits prevent the common mistake of developing with one Python version and deploying or sharing with another, where differences in language features or library behaviour cause failures that are hard to trace.

Finally, be cautious about uninstalling old Python versions, because system tools or other projects may depend on them, and removing a Python that an environment was based on can break that environment, even if the environment folder itself remains. An environment links to its base Python, and if that base is removed, the environment may no longer work, requiring recreation from a different base. Before removing a Python version, check which environments depend on it, and whether any system tools need it, and recreate affected environments from a remaining Python version if necessary, after testing that the project works with the new version. Managing multiple Pythons, like managing multiple package sets, is about explicitness: know what you have, know what each project needs, and make choices deliberately, rather than relying on defaults that may change without warning.

Check: Why is a virtual environment tied to the Python version it was created from, and why does that matter?

Section 25: Verifying Which Python and pip You Are Using

Verifying which Python and pip you are using is a quick check that prevents many environment mistakes, and it should become a reflex whenever you start work, switch projects, or encounter an unexpected import error. The check involves asking your shell which python and which pip it will run, and confirming that the paths point inside the active environment's folder, rather than to the system installation or to another environment. If the paths point to the expected environment, you are working in the right place, and you can install packages or run code with confidence. If they point elsewhere, you need to activate the correct environment, or deactivate and reactivate, before proceeding, because continuing in the wrong environment will install packages in the wrong place or run code with the wrong dependencies.

Verifying also involves checking versions, because the right environment with the wrong Python version, or the wrong pip version, can still cause problems. Checking the Python version confirms that the environment was created from the intended Python, and checking the pip version confirms that pip itself is up to date enough to handle the packages you need, though pip updates are usually straightforward. Version checks are especially important when setting up a project on a new computer, or after updating Python, because the default Python may have changed, and an environment created earlier may now use a different version than you expect. A quick version check, compared with the project's recorded Python version, catches such mismatches early, before they cause confusing failures during development or testing.

Make verification part of your workflow, not just a troubleshooting step. When you activate an environment, glance at the prompt for the environment name, and occasionally verify the paths explicitly, especially after opening a new terminal, after switching projects, or after a system update that might have changed Python installations. That habit takes seconds, and it prevents the frustrating experience of installing a package, running code, and discovering that the package is not found, because it was installed in a different environment, or that code behaves differently, because it ran with a different Python. Verification is a small discipline with a large payoff, because environment mismatches are common, easy to make, and easy to prevent with a quick check.

Finally, teach verification to collaborators, by including it in setup instructions, so that new team members also check their environment before working. A setup instruction that says activate the environment and verify that Python points inside it, with the command to do so, is more helpful than one that merely says activate the environment, because it gives the newcomer a way to confirm success, rather than assuming it. Verification turns activation from a hope into a checked fact, and that checking culture, applied to environments as well as to code through tests, is a hallmark of reliable development. When in doubt, verify: it is faster than debugging a problem caused by working in the wrong environment.

Check: What should you check to confirm you are working in the intended virtual environment?

Section 26: pip Configuration and Caches

pip uses a cache to store downloaded packages, so that reinstalling the same package and version does not require downloading it again, which saves time and bandwidth, especially when recreating environments or installing on a slow connection. The cache is managed automatically, and you generally do not need to interact with it, but understanding it helps you interpret pip's behaviour: an installation that completes quickly may have used the cache, while a first installation downloads from the index. The cache can grow large over time, as it accumulates packages from many projects, and pip provides commands to inspect and clear it, though clearing it is rarely necessary unless disk space is a concern. Knowing that a cache exists prevents confusion when an installation succeeds without network access, because the package was already cached, or when a package appears to install instantly, because it was retrieved locally rather than downloaded.

pip also has configuration options, which can be set in configuration files, that control aspects of its behaviour, such as which package index to use, whether to use a cache, and how to handle certain situations. For most learners, the default configuration is sufficient, and changing it is unnecessary, but for organisations or for specific situations, such as using a private package index for internal packages, configuration becomes important. A private index, which hosts packages that are not public, allows a team to share internal packages securely, without publishing them publicly, and pip can be configured to use such an index in addition to, or instead of, the public one. That configuration, while beyond the scope of a first project, illustrates that pip is flexible, and that its behaviour can be adapted to different needs, through configuration rather than through different tools.

Configuration files live in standard locations, depending on the operating system and on whether the configuration is global, user-level, or project-level, and they use a simple format for setting options. Misconfigured pip, such as a configuration that points to an unavailable index, can cause installation failures that seem mysterious, because pip will fail to find packages that are actually available publicly. Most installation failures, however, are not configuration problems, but dependency conflicts, network issues, or environment mismatches, so check those first, and consider configuration only if the basic checks do not explain the failure.

Finally, keep pip itself reasonably up to date, because newer versions fix bugs, improve dependency resolution, and support newer package formats, while older versions may fail to install packages that use newer features. Updating pip is usually done within an environment, so that each environment has its own pip version, though the base pip used to create environments also matters. Updating pip is a low-risk operation, but it should still be done deliberately, and followed by a test that installations still work, because a pip update, like any tool update, can occasionally change behaviour. For most projects, pip's defaults, its cache, and occasional updates are sufficient, and the key habits are to understand that a cache exists, that configuration can affect behaviour, and that pip, like any tool, should be kept reasonably current and checked when installations behave unexpectedly.

Check: How does pip's cache save time when recreating environments, and when might you clear it?

Section 27: Environments for Different Kinds of Projects

Different kinds of projects have different environment needs, and recognising those differences helps you set up environments appropriately, rather than using a one-size-fits-all approach. A small script, used only by you on one computer, may need only a simple environment, or even no environment, if it uses only the standard library, because its dependencies are minimal and its portability requirements are low. A larger application, shared with a team or deployed to users, needs a carefully managed environment, with a complete requirements file, a frozen file for deployment, and verified recreation, because its correctness depends on exact dependencies and because many people rely on it. The environment's rigour should match the project's importance and its audience, not exceed it unnecessarily, nor fall short where reliability matters.

A learning project, such as the exercises in this book, benefits from a simple environment workflow, focused on learning the concepts rather than on production rigour. Creating an environment, activating it, installing a few packages, and recording them in a requirements file is sufficient to learn isolation, installation, and recording, without the complexity of freezing, private indexes, or deployment pipelines. As you progress to larger projects, you can add rigour gradually: freezing for deployment, testing recreation, and managing updates more formally, building on the simple workflow rather than replacing it entirely. That gradual progression, from simple to rigorous as projects grow, is a sensible way to learn environment management, because it introduces complexity only when it is needed to solve a real problem, not in advance of one.

A data or scientific project may have many dependencies, including large packages for data handling and visualisation, and its environment may be large and slow to recreate. For such projects, environment management is especially important, because dependency conflicts are more likely with many packages, and because reproducibility is critical for scientific work, where results must be recreatable from the same code and data. Careful version recording, freezing, and documentation of the Python version and the environment setup are essential for such projects, even if they add overhead, because the cost of irreproducibility, results that cannot be verified, is far higher. Data projects are a case where rigorous environment practice pays off most clearly, and where the habits from this chapter should be applied most thoroughly.

A web project, covered in later chapters, may have dependencies for its framework, its database handling, and its deployment, and its environment must be recreatable on a server, which may have a different operating system or Python version from the development computer. That deployment requirement makes reproducibility and testing recreation especially important, because a web project that works locally but cannot be set up on a server is not deployable. Environment documentation for a web project should therefore include server-specific setup notes, such as Python version and system dependencies, in addition to the usual requirements file. Recognising a project's kind, and its specific environment needs, helps you tailor your workflow appropriately, rather than applying the same minimal workflow to every project, regardless of its importance or its deployment context.

Check: Why should a deployed application have more rigorous environment management than a small personal script?

Section 28: Sharing Environments in a Team

Sharing environments in a team means sharing the information needed to recreate them, not sharing the environment folders themselves, as discussed earlier. That information includes the requirements file, the Python version, setup instructions, and any notes about system dependencies or configuration that affect the environment. When a new team member joins, they follow those instructions to create their own environment, which will be functionally equivalent to others', even though its folder and paths will differ. That equivalence, same packages and versions, different local paths, is sufficient for collaboration, because code depends on packages and versions, not on the exact folder path, provided the code does not hard-code paths into the environment folder, which it should not. Sharing recipes, not environments, is therefore the team standard, and it works because environments are recreatable from recipes.

Team environment management also requires coordination when dependencies change. When one team member updates a package, they should update the requirements file, test the project with the new version, and communicate the change to the team, so that others can update their environments accordingly, by reinstalling from the updated requirements file or by updating the specific package. Without communication, team members' environments will drift apart, with some using old versions and others using new ones, leading to bugs that appear for some team members but not others, which are confusing and time-consuming to debug. Communication can be as simple as a message in the team's channel, or a note in a change description, stating which package changed, to which version, and why, so that others know to update. That small communication habit prevents a large class of team environment problems.

Teams should also agree on conventions for environment management, such as the folder name for environments, the name and location of the requirements file, the Python version to use, and the process for updating dependencies. Conventions reduce confusion, because every team member sets up projects the same way, and troubleshooting becomes easier, because environments are structured consistently across the team. A new member who knows the team's conventions can set up any team project without needing project-specific instructions for basic steps, because the steps are the same everywhere. Document those conventions in a team guide or in each project's setup instructions, so that they are visible and can be followed, rather than relying on oral tradition that new members may miss.

Finally, teams should periodically verify that environments can still be recreated from shared files, by having a team member, preferably one who did not set up the project originally, follow the setup instructions from scratch and run the project's tests. That verification, done occasionally, catches drift, omissions, and outdated instructions before they affect a new hire or a deployment, where they would be more costly. It also gives the team confidence that their shared recipe works, not just that it worked once, when the project was created. Team environment sharing is therefore not just about files, but about communication, conventions, and periodic verification, all of which together ensure that every team member works with an equivalent, up-to-date environment, and that the project remains reproducible as it evolves.

Check: Why should a team share requirements and setup instructions, rather than environment folders?

Section 29: Updating Dependencies Safely

Updating dependencies safely is a process, not a single command, because updates can introduce breaking changes, even when they are intended only to fix bugs. A safe update process starts with a working, tested baseline: run the project's tests before updating, and confirm they pass, so that you know the starting point is good. Then update one package at a time, as discussed earlier, rather than updating everything at once, so that if tests fail after an update, you know which package caused the failure. After each update, run the tests again, and if they pass, record the new version in the requirements file, so that the update is captured. If they fail, investigate the failure, decide whether to fix your code to work with the new version, or to revert to the old version and postpone the update, and document that decision. That one-at-a-time, test-after-each process is slower than bulk updating, but it is far safer, because it isolates each change and verifies it before proceeding.

Before updating, it is also useful to check what changed in the new version, by reading its release notes or changelog, if available, which usually summarise breaking changes, new features, and bug fixes. Release notes help you anticipate whether an update is likely to affect your project, and they help you understand test failures if they occur, by pointing at intentional changes that may require code adjustments. Not all packages provide detailed release notes, and some changes may be undocumented, which is why testing remains essential even after reading notes. Make reading release notes a habit for significant updates, especially major version updates, where breaking changes are most likely.

Updating in a disposable environment, as discussed for experiments, is another safety practice. Create a copy of your environment, or a fresh environment from the current requirements, update the package there, and run tests, without touching your working environment. If the update works, apply it to your working environment and update the requirements file. If it fails, discard the disposable environment, and your working environment remains intact, allowing you to continue work while you investigate the update at leisure. That separation between experimentation and working setup prevents updates from blocking your work, and it encourages more frequent, smaller updates, because the risk of trying an update is low. Disposable environments make safe updating practical, by removing the fear that an update will break a working setup irreparably.

Finally, schedule dependency updates regularly, rather than letting them accumulate for months or years, because accumulated updates are harder to apply safely: many packages changing at once makes it difficult to isolate failures, and the gap between versions may include multiple breaking changes that interact in complex ways. Regular, small updates, such as reviewing dependencies monthly or when a security update is announced, keep the project current with less effort per update, and reduce the risk of being stuck on an old, unsupported version that can no longer be updated cleanly. Regular updating, done safely with tests and one-at-a-time changes, is a maintenance habit that keeps projects healthy over time, rather than a periodic crisis when updates can no longer be postponed.

Check: What is a safe step-by-step process for updating a dependency?

Section 30: When Environments Go Wrong

Environments can go wrong in several ways, and recognising the symptoms helps you choose the right fix, rather than guessing. A common symptom is an import error for a package you believe is installed. That usually means the wrong environment is active, the package was installed in a different environment, or the package's import name differs from its installation name. The fix is to verify the active environment, list installed packages in that environment, and check the import name, as discussed earlier. Another symptom is a version conflict, where pip reports that packages require incompatible versions. The fix is to resolve the conflict, as discussed earlier, by finding compatible versions or simplifying dependencies. A third symptom is a corrupted environment, where activation fails, or Python itself fails to run from the environment, perhaps because files were deleted or permissions changed. The fix is usually to recreate the environment from the requirements file, which is faster and more reliable than trying to repair a corrupted folder file by file.

Another kind of environment problem is drift, where an environment's installed packages no longer match its requirements file, because packages were installed or removed without updating the file, or because the file was updated without reinstalling. Drift causes recreation failures, where a fresh environment built from the file does not work, even though the original environment does, and it causes confusion when collaborators' environments differ. The fix for drift is to reconcile the environment and the file: list installed packages, compare with the file, and update whichever is out of date, either by installing missing packages or by recording installed ones, after verifying that each package is actually needed. Then verify by recreating the environment from the updated file and running tests, to confirm reconciliation was successful. Drift is prevented by the habit of recording immediately, as discussed earlier, but when it occurs, reconciliation plus verification restores consistency.

Performance problems can also be environment-related, though less commonly. If a program is slow, check first whether the slowness is in the code, through profiling, as discussed in the debugging chapter, before blaming the environment. An environment problem that does affect performance is using the wrong Python version, or a Python built without optimisations, but such cases are rare for learners and usually involve system-level Python installations, not project environments. Focus environment troubleshooting on correctness, meaning imports, versions, and activation, rather than on performance, unless measurement points at the environment as the cause.

Finally, when an environment problem resists troubleshooting, do not hesitate to recreate it. Recreation is a powerful reset: it discards any corruption, drift, or experimental clutter, and rebuilds a clean environment from the recorded requirements, which, if accurate, will work. If recreation also fails, the problem is likely in the requirements file, in the Python version, or in the project's code, not in the environment itself, and your investigation should shift to those areas. That shift, from environment to requirements or code, is itself diagnostic progress, because it narrows the problem. Environments are meant to be disposable, and treating them as such, by recreating freely when they go wrong, rather than nursing a broken environment indefinitely, is a healthy attitude that will save you time and frustration throughout your development career.

Check: What are three common environment problems, and what is the appropriate fix for each?

Section 31: pip and the Package Index

The package index is the public repository from which pip downloads packages, and understanding it helps you understand where packages come from and how to evaluate them. The index hosts thousands of packages, each with multiple versions, documentation, and metadata, including its author, its licence, its dependencies, and its description. When you run an install command, pip contacts the index, finds the package and the version you requested, or the newest compatible version if you did not specify one, downloads the package files, and installs them. That process, which feels instantaneous, involves a large, distributed system that serves packages to developers worldwide, and it is a remarkable resource that makes Python's ecosystem so rich. Appreciating the index as infrastructure, not just as a download source, helps you use it responsibly, by choosing packages thoughtfully and by not abusing it with unnecessary downloads.

Browsing the index, through its website, is a useful way to evaluate a package before installing it, because the package's page typically shows its description, its documentation links, its version history, its dependencies, and sometimes its download statistics or its maintenance status, though statistics should be interpreted cautiously, as discussed in the task constraints, and popularity alone is not proof of quality. A package page that is well-documented, with clear examples and a recent release history, is a good sign, while a page with no documentation, no recent releases, and vague descriptions is a warning. That comparison, done before installation, saves the effort of switching packages later, after code has been written against the first choice.

The index also hosts different kinds of package files, including source distributions, which contain the package's source code and need to be built during installation, and pre-built distributions, often called wheels, which contain ready-to-install files and install faster, without needing a build step. pip prefers pre-built distributions when they are available and compatible with your Python version and platform, because they are faster and more reliable to install, falling back to source distributions when necessary, which may require additional tools or system libraries to build. Understanding that distinction helps you interpret installation messages: an installation that builds from source may take longer and may fail if build tools are missing, while an installation from a pre-built distribution is usually quick and straightforward. For most packages, pip handles this choice automatically, and you need not intervene, but knowing the difference explains why some installations are fast and others are slow or complex.

Finally, the index is a community resource, maintained by volunteers and funded by donations and sponsors, and using it responsibly includes not overloading it with unnecessary requests, such as repeatedly installing the same packages in a loop, and respecting its terms of use. For most developers, responsible use is automatic, because pip's cache reduces repeated downloads, and normal development does not strain the index. The broader lesson is that Python's ecosystem depends on shared infrastructure, maintained by the community, and that infrastructure works because developers use it thoughtfully, contribute packages and documentation, and report problems constructively. As you become a Python developer, you benefit from that ecosystem, and, in time, you may contribute to it yourself, by publishing a package, improving documentation, or helping maintain a project, continuing the cycle that makes the ecosystem valuable for everyone.

Check: What information can a package's index page give you before you decide to install it?

Section 32: Installing from Other Sources

While the public package index is the usual source for pip installations, pip can also install packages from other sources, such as a local folder, a version control repository, or a direct file, which is useful in specific situations. Installing from a local folder is useful when you are developing a package yourself, or when you have a package's source code locally and want to install it into an environment for testing, without publishing it to the index. Installing from a version control repository is useful when you need a version of a package that has not yet been released to the index, such as a fix that exists only in the repository's latest code, though using unreleased code carries risk, because it has not undergone the same testing as a release, and it may change without notice. Installing from a direct file is useful when you have a package file, perhaps provided by a colleague or downloaded separately, and want to install it without using the index.

Each of these sources has trade-offs compared with the index. Local and repository installations are less reproducible, because they depend on files or repository states that may change, or that are not available to collaborators who do not have the same local files or repository access. They are also harder to version, because a local folder or a repository branch does not have a fixed version number in the same way a released package does, making it difficult to record exactly what was installed in a requirements file. When you do use them, document the source, the reason, and the version or commit used, so that collaborators can reproduce the setup, or understand why it cannot be reproduced exactly from the index alone.

A related concept is installing a package in editable mode, where pip installs a package by linking to its source folder, rather than copying its files, so that changes to the source are immediately reflected in the installed package, without reinstalling. Editable installation is valuable when developing a package, because it lets you test changes immediately, and when working on a project that is structured as a package, because it lets you import the project as if it were installed, while still editing its source directly. Understanding editable mode helps you recognise it when you encounter it in project instructions, and helps you use it appropriately during development, without confusing it with a normal installation.

Finally, be cautious about installing from untrusted sources, for the same safety reasons discussed earlier. A package file from an unknown source, or a repository you have not vetted, could contain malicious code, just as a package from the index could, but with even less scrutiny, because it has not passed through the index's checks or community review. Only install from other sources when you trust the source, understand why the index version is insufficient, and have documented the decision. For most projects, the index is sufficient, and other sources are exceptions for specific development needs, not routine practice. Keeping installations primarily from the index, with other sources used deliberately and documented, maintains reproducibility and safety, while still allowing flexibility when it is genuinely needed.

Check: Why are installations from a local folder or repository less reproducible than installations from the package index?

Section 33: Environment Variables and Configuration

Environment variables are settings stored in your operating system's environment, which programs can read to adapt their behaviour without changing code, and they interact with virtual environments in a few important ways. Activation itself works by changing environment variables, notably the path variable that determines where the shell looks for commands, so that the environment's scripts folder is found first. Other environment variables may control Python's behaviour, such as which directories it searches for modules, or how it handles certain features, and those variables can affect a program's behaviour in ways that are not visible in its code, which can cause confusion when a program works in one session but not in another, because the variables differ between sessions. Understanding that programs can depend on environment variables, in addition to code and packages, is important for debugging environment-related problems.

For project configuration, environment variables are often used to store settings that vary between environments, such as whether a program is running in development or in production, or where it should find its data files, without hard-coding those settings into code. That practice keeps code portable, because the same code can run in different settings by changing variables, rather than by editing code, and it keeps sensitive settings, such as passwords or keys, out of code, where they might be accidentally shared. However, this book's guidance, consistent with the user's preference for built-in tools and against pasting keys, is to avoid requiring users to supply their own keys, and to use environment variables only for non-sensitive configuration, such as mode or paths, not for secrets that should be managed through secure, approved flows. Environment variables are a configuration mechanism, not a secure vault, and they should not be treated as a safe place for sensitive credentials, because they can be visible to other processes or recorded in logs if handled carelessly.

When debugging, check environment variables if a program behaves differently across sessions or computers, because a variable set in one session, perhaps by activation, by a startup script, or manually, may persist or may be absent in another session, causing a difference. Listing relevant variables, and comparing them between a working and a failing session, can reveal such a difference quickly, especially for variables that affect Python's module search or a program's configuration. Document which variables your project uses, if any, in its setup instructions, so that collaborators know to set them, and so that a new setup can be verified by checking those variables, rather than by guessing. That documentation, like requirements files, makes a project's setup reproducible, by capturing not just packages, but also configuration that affects behaviour.

Finally, do not rely on environment variables for essential project setup in a learning project, unless you have a clear reason, because they add a hidden dependency that is easy to forget and hard to debug. Use environment variables when they genuinely help, such as for distinguishing development from production, or for settings that must vary without code changes, and document them when you do, so that their effect is visible to future readers. That balanced use, variables for varying configuration, code and files for stable configuration, keeps projects both flexible and understandable, without introducing hidden dependencies that undermine reproducibility.

Check: How can environment variables cause a program to behave differently in different terminal sessions?

Section 34: Cleaning Up Old Environments

Old environments accumulate over time, as projects are finished, abandoned, or superseded, and cleaning them up periodically is good hygiene, because each environment uses disk space, sometimes significant space if it contains large packages, and because old environments can cause confusion, when you forget which environment belongs to which project, or when you accidentally activate an old environment for a new project. Cleaning up means deleting environment folders for projects you no longer work on, after confirming that the project's code and requirements file are safely stored elsewhere, such as in version control, so that the environment can be recreated if the project is ever revived. Deleting an environment is safe, provided the project's recipe, its requirements and instructions, is preserved, because the environment itself is recreatable, while the recipe is the valuable, irreplaceable part.

Before deleting an environment, verify that it is indeed the environment you intend to delete, and not a project folder containing code, by checking its name, its location, and its contents, as discussed for recreation. Also check whether the environment is currently active in any terminal session, because deleting an active environment can cause confusing errors in that session, though it will not harm the system. Deactivate first, then delete, and then confirm deletion by listing the project folder to ensure the environment folder is gone and code remains. That careful sequence, deactivate, verify, delete, confirm, prevents the distressing mistake of deleting the wrong folder, and it is a habit worth practising, especially when using commands that delete folders permanently, without a trash or undo step, depending on your system.

Cleaning up also includes reviewing disk usage, to identify which environments are largest and whether they are still needed. Large environments, often those for data or web projects with many packages, may use hundreds of megabytes each, and several such environments can consume gigabytes, which may matter on a computer with limited storage, such as a laptop or a phone-based development setup. Listing environments by size, and deleting those for finished projects, frees space for current work, and it reduces clutter that makes it harder to find current environments. Keep a simple list, perhaps in a note, of active projects and their environments, so that you can distinguish current from old at a glance, without needing to inspect each folder's contents to remember which project it belongs to.

Finally, cleaning up is not just about disk space; it is about mental clarity. A workspace with many old environments, old projects, and forgotten folders is harder to navigate, and it increases the chance of working in the wrong environment or with outdated code. Periodic tidying, deleting old environments, archiving finished projects, and updating your project list, keeps your workspace focused on current work, which improves productivity and reduces mistakes. Make cleaning up a periodic habit, perhaps when you finish a project or at the start of a new one, rather than letting clutter accumulate until it becomes overwhelming. A tidy workspace, like a tidy environment, is easier to work in, easier to share, and easier to maintain, and it reflects the same discipline that makes dependency management itself effective: keep what you need, record how to recreate it, and discard what you no longer need, safely and deliberately.

Check: Why is it safe to delete an old environment folder, provided the project's requirements file is preserved?

Section 35: Environments and Editors

Code editors and development tools interact with virtual environments, because they need to know which Python to use for running code, checking errors, and providing completions, and using the wrong Python can cause an editor to report missing packages or to run code with the wrong dependencies, even though the terminal works correctly. Most editors let you select a Python interpreter, and you should select the interpreter inside your project's active environment, rather than the system Python, so that the editor's checks and runs match your terminal's behaviour. That selection is usually per-project or per-workspace, so that each project uses its own environment automatically when you open it, without needing to reselect each time. Learning how to select an interpreter in your editor, and verifying that selection, is an important part of environment workflow, because an editor using the wrong Python can produce confusing errors that seem like code problems but are actually environment mismatches.

Editor integration also affects debugging, because an editor's debugger, if it uses the wrong Python, will run code with the wrong packages, potentially hiding or introducing bugs that do not occur in the terminal. Similarly, an editor's test runner, if it uses the wrong Python, may report failures that do not occur when tests are run from the terminal in the correct environment, or vice versa. To avoid such mismatches, ensure your editor's Python, its test runner, and its debugger all point to the same environment you use in the terminal, and verify that consistency when you first set up a project in an editor, and again if you notice discrepancies between editor and terminal behaviour. Consistency across tools, terminal, editor, debugger, and test runner, is the goal, because inconsistent tools produce inconsistent results that waste debugging time.

Some editors automatically detect virtual environments in a project folder, by looking for conventional environment folder names, and offer to use them, which simplifies setup, provided you use a conventional name. Using a conventional environment folder name, as discussed earlier, therefore helps not only human collaborators, who recognise the folder as an environment, but also tools, which can detect it automatically. If your editor does not detect an environment, you can usually configure it manually, by pointing it to the environment's Python executable, which lives in the environment's scripts or binaries folder. That manual configuration, once done per project, persists, so you need not repeat it each session, but you should verify it after recreating an environment, because recreation may change the interpreter's path, especially if the environment folder's location changed.

Finally, editor settings for Python, like other project settings, should be distinguished between personal settings, which belong to your editor configuration and should not be shared, and project settings, which define how the project should be set up and can be shared, such as the Python version or the environment location pattern. Sharing personal editor settings can cause confusion for collaborators who use different editors or different paths, while sharing project-relevant settings, through documentation rather than through editor-specific files, helps collaborators set up their own editors correctly. Document the project's Python version and environment setup in its instructions, so that collaborators can configure any editor to use the correct environment, rather than relying on editor-specific files that may not transfer between different tools or setups.

Check: Why should your editor use the same Python interpreter as your project's virtual environment?

Section 36: Common Environment Mistakes

Common environment mistakes are worth listing explicitly, because recognising them in advance helps you avoid them, and recognising them during debugging helps you fix them quickly. The first mistake is installing packages without activating an environment, so that they go into the system Python or into the wrong environment, and then wondering why code cannot find them when run in the intended environment. The fix is to activate first, verify, and then install, and the prevention is to make activation and verification a habit at the start of every work session. The second mistake is forgetting to record an installation in the requirements file, so that a fresh environment built from the file is missing a package, even though the original environment works. The prevention is to record immediately, as part of installation, not later from memory.

The third mistake is using the same environment for multiple projects, to save time, which defeats the purpose of isolation and eventually causes version conflicts, as projects' needs diverge. The prevention is to create a separate environment per project, even for small projects, because creation is cheap and isolation's benefits outweigh the small overhead. The fourth mistake is tracking the environment folder in version control, which bloats history and causes conflicts, as discussed earlier. The prevention is to set up an ignore file early, excluding the environment folder, and to verify that version control status does not show the environment as untracked or tracked. The prevention is to ensure requirements are recorded and, ideally, version-controlled, before deleting any environment.

The sixth mistake is upgrading all packages at once, without testing, and then facing multiple failures that are hard to attribute to specific upgrades. The prevention is to upgrade one at a time, with tests after each, as discussed earlier. The seventh mistake is ignoring Python version differences, by creating an environment from the default Python without checking whether the project needs a specific version, leading to failures when code uses features from a different version. The prevention is to record the Python version and to create environments from that version deliberately. The eighth mistake is working in an environment that is active in one terminal window but not in another, and being confused when behaviour differs between windows. The prevention is to understand that activation is per-session, and to activate in each window where you work, verifying as needed.

Recognising these mistakes as patterns, rather than as isolated incidents, helps you develop a checklist for environment work: activate, verify, install and record, test, deactivate, and periodically recreate to verify. That checklist, followed consistently, prevents most environment mistakes, and when a mistake does occur, the checklist helps you diagnose it, by asking which step was skipped. For example, a missing package suggests installation or recording was skipped, a version conflict suggests isolation or upgrading discipline was skipped, and a tracked environment folder suggests ignore setup was skipped. Checklists are not bureaucracy; they are memory aids for habits that are easy to forget under pressure, and they are especially valuable for environment management, where mistakes are common, easy to make, and sometimes time-consuming to fix, but also easy to prevent with a few consistent steps.

Check: Which environment mistake causes a fresh environment built from requirements to miss a package that works in the original?

Section 37: Environment Workflow for the Capstone

The capstone project for this book, described in the outline as PyDay, a command-line task-and-notes manager using only the standard library, has a deliberately simple environment story, which makes it an ideal example for applying this chapter's lessons without complexity. Because PyDay uses only the standard library, it needs no external packages, and therefore its environment, while still useful for isolation and for practising workflow, does not need a requirements file with external dependencies, or a frozen file, because there is nothing to install beyond the standard library itself. That simplicity is intentional: it lets learners focus on the capstone's logic, structure, and commands, without the distraction of dependency management, while still practising environment habits, such as creating and activating an environment for the project, even if that environment remains empty of external packages.

Even for a standard-library-only project, creating an environment is worthwhile, because it establishes the habit and because it isolates the project's Python version, ensuring that PyDay runs with the Python version it was developed with, even if the system Python changes. It also provides a clean context for running tests, as covered in the previous chapter, where tests run in the project's environment, with the project's Python, without interference from globally installed packages that might mask a missing standard library import or a version difference. Practising environment creation for PyDay, even though its requirements are minimal, reinforces the workflow so that it becomes automatic for later projects that do have external dependencies, where the workflow matters more. Habits formed on simple projects transfer to complex ones, while habits neglected on simple projects are harder to adopt under the pressure of a complex project.

For PyDay, the environment workflow can be documented simply: create an environment, activate it, run the program and its tests from within it, and deactivate when done. No installation steps are needed, beyond the environment itself, and no requirements file with external packages is needed, though a note stating that the project uses only the standard library is useful documentation, because it tells collaborators that no installation is required, which is itself valuable information. That note prevents a collaborator from searching for a requirements file that does not exist, or from assuming dependencies are missing, and it makes the project's simplicity explicit, rather than leaving it ambiguous whether dependencies were forgotten or intentionally omitted. Explicit simplicity, documented as a choice, is better than implicit simplicity, which looks like an oversight.

As you build PyDay in later chapters, use its environment consistently, even though it is simple, and notice how the workflow feels when there are no packages to manage. That experience, of an environment as a clean, isolated context for running and testing, is the foundation for understanding environments in more complex projects, where the same workflow, plus installation and recording, handles external dependencies. When you later work on projects with many dependencies, you will appreciate having learned the workflow on a simple project, where mistakes are low-stakes and easy to correct, rather than learning it for the first time on a complex project, where a mistake could break a shared setup. PyDay's simplicity is therefore not a limitation, but a pedagogical choice, letting you master environment habits before dependency complexity is added.

Check: Why is creating an environment still worthwhile for a project that uses only the standard library?

Section 38: Documenting Setup for Others

Documenting setup for others means writing instructions that let someone else, who may be unfamiliar with your project, set it up and run it successfully, using only your documentation and the project's files. Good setup documentation starts with prerequisites: what Python version is needed, what operating system considerations apply, and what tools must be installed, such as a version control tool or an editor, if relevant. It then gives step-by-step instructions: how to obtain the code, how to create an environment, how to activate it, how to install dependencies, and how to verify the setup, by running tests or a main command and checking that the output matches what is expected. Each step should be concrete, with the command or action to perform, not vague, such as set up the environment, without saying how, because vague instructions assume knowledge the reader may not have.

Setup documentation should also include troubleshooting for common problems, such as what to do if activation fails, if an import error occurs, or if a package installation fails, with pointers to the relevant sections of this chapter or to the project's issue tracker. Troubleshooting sections save time for both the newcomer and the maintainer, because they answer frequent questions without requiring a personal response each time, and they show that common problems have been anticipated. Keep troubleshooting brief, focusing on the most common issues for your project, rather than trying to cover every possible failure, which would make the documentation unwieldy. A short troubleshooting list, with clear symptoms and fixes, is more useful than a long, exhaustive list that is hard to scan.

Documentation should be tested, by having someone else, or yourself in a fresh setting, follow it exactly, without using prior knowledge to fill gaps, and noting where they get stuck or confused. That test, often called a fresh-eyes test, reveals omissions, ambiguities, and assumptions that the author, who knows the project well, cannot see, because they fill gaps unconsciously. Fix the documentation based on that test, and repeat it if the project changes significantly, so that documentation stays accurate as the project evolves. Documentation that has never been tested by a fresh reader is a draft, not a finished guide, and it will likely fail the first real newcomer who tries to use it, causing frustration and support burden that testing would have prevented.

Finally, keep setup documentation close to the project, in a file that is version-controlled alongside code, so that it evolves with the project and is available to anyone who obtains the code. A common name for such a file is a readme, which typically includes a project description, setup instructions, usage instructions, and contribution notes, among other sections. A readme that includes clear setup instructions, verified by a fresh-eyes test, and maintained alongside code changes, is a professional asset, signalling that the project is welcoming to collaborators and serious about reproducibility. For a learner, writing a good readme for a small project is excellent practice, because it forces you to think about your project from a newcomer's perspective, which improves both the documentation and the project itself, by revealing setup complexities that could be simplified.

Check: Why should setup documentation be tested by someone following it exactly, without prior knowledge?

Section 39: Environments and Deployment Basics

Deployment means making a program available to run in a place other than your development computer, such as a server, a colleague's computer, or a user's device, and environments play a central role in deployment, because the deployment environment must match the development environment closely enough for the program to run correctly there. The basic deployment workflow, for a Python project, involves creating an environment on the target machine, installing dependencies from the project's requirements or frozen file, copying the project's code to the target, and running the program or its tests there to verify the setup. That workflow mirrors local setup, but with additional considerations, such as the target's operating system, Python version, available system libraries, and permissions, which may differ from development and must be accounted for in deployment instructions. Deployment is covered in more detail in a later chapter, but its environment foundations belong here, because a project that cannot be set up on a target cannot be deployed, regardless of how well its code works locally.

A key deployment consideration is that the target environment should be built from recorded requirements, not copied from development, for the same reasons discussed for sharing: copied environments are machine-specific and may not work on a different system, while recreated environments, built for the target's system and Python version, are more likely to work, provided the requirements are accurate and the Python version is compatible. Freezing, discussed earlier, is particularly valuable for deployment, because it ensures the target uses exactly the versions tested during development, rather than newer versions that might behave differently. That exactness reduces deployment surprises, where a program works locally but fails on the target due to a version difference, which is a common and frustrating deployment problem that freezing helps prevent.

Deployment environments should also be minimal and clean, containing only what the program needs to run, not development tools, test packages, or experimental packages that were used during development but are not needed in operation. Distinguishing between runtime dependencies, needed to run the program, and development dependencies, needed only to develop or test it, and recording them separately, is a practice that supports minimal deployment environments, though for small projects, a single requirements file may be sufficient, provided it does not include unnecessary packages. As projects grow, separating runtime and development dependencies becomes more important, and it is a habit worth adopting when the distinction becomes clear.

Finally, deployment should be verified, by running the program's main functionality or its tests in the deployment environment, before considering deployment complete, just as local setup should be verified by running tests. That verification catches environment mismatches, missing system dependencies, and configuration differences that are specific to the target, and it gives confidence that deployment actually works, not just that files were copied. Document the deployment environment setup, including Python version, requirements file used, and any target-specific steps, so that future deployments can be repeated reliably, and so that problems can be diagnosed by comparing the documented setup with the actual target setup. Deployment, like development, depends on environments, and the habits from this chapter, isolation, recording, recreation, and verification, are the foundation for successful deployment, regardless of the target's scale or complexity.

Check: Why should a deployment environment be built from requirements on the target, rather than copied from development?

Section 40: Reviewing Your Environment Habits

Reviewing your environment habits periodically helps you improve them, because habits that are not reviewed tend to drift, with shortcuts creeping in, such as installing without recording, or working without activating, that undermine the benefits of environments. A useful review asks a few questions: for each active project, does it have its own environment, is that environment's requirements file up to date, can the environment be recreated from that file, and is the environment excluded from version control? Answering those questions, project by project, reveals gaps quickly, and fixing them, by updating requirements, recreating to verify, or adding ignore patterns, restores good practice before gaps cause problems. That review, done occasionally, perhaps when finishing a project or starting a new one, is a small investment that keeps environment management healthy across all your projects, not just the one you are currently focused on.

A review should also consider whether your environment workflow is still appropriate for your projects' current scale. A workflow that was sufficient for small learning projects, with a single requirements file and manual updates, may need more rigour for larger projects, with frozen files, separated development dependencies, or automated recreation checks. Conversely, a workflow that is overly rigorous for a small project, with complex freezing and deployment steps for a script used only by you, may be wasting effort that could be spent on code. Adjusting workflow rigour to project scale, increasing it as projects grow in importance and audience, and simplifying it when a project is small or personal, is a sign of mature judgement, not of inconsistency. Review is the moment to make those adjustments deliberately, rather than letting workflow evolve haphazardly through habit alone.

Reviewing also involves learning from environment problems you encountered, by noting what went wrong, what fixed it, and what habit would have prevented it, in a notebook or a brief note, similar to the debugging notebook from the previous chapter. For example, if you encountered a missing package in a recreated environment, the lesson is to record installations immediately, and the habit to reinforce is immediate recording. If you encountered a version conflict, the lesson may be to simplify dependencies or to update more regularly, and the habit to reinforce is periodic review of dependencies. Those lessons, recorded and reviewed, turn environment problems into improvements, rather than into repeated frustrations, and they build expertise over time, so that environment management becomes increasingly smooth and reliable.

Finally, share your environment habits with others, by documenting them in project readmes, by helping collaborators set up environments, and by discussing environment practices in team settings, because environment management is a shared skill, and teams benefit when everyone follows consistent, good practices. Sharing also exposes your habits to feedback, where others may suggest improvements, such as a better folder naming convention, a useful verification step, or a tool that simplifies recreation. That feedback loop, reviewing, learning, sharing, and improving, is how individual habits become team standards, and how team standards evolve to meet new challenges. Environment management, like testing and debugging, is a practice that improves with reflection, and periodic review is the mechanism for that improvement, ensuring that your environments remain clean, reproducible, and ready for whatever projects come next.

Check: What four questions should a periodic environment review ask about each active project?

Section 41: Environment Checklists

A checklist for environment work turns the habits from this chapter into a concrete, repeatable sequence that you can follow without relying on memory, especially when you are tired, in a hurry, or setting up an unfamiliar project. A creation checklist might include: choose a project folder, decide the Python version, create the environment with a conventional name, activate it, verify the Python path, create an empty requirements file, and add an ignore pattern for the environment folder to version control. Following that checklist for each new project ensures no step is missed, such as forgetting the ignore pattern, which would cause the environment to appear in version control status, or forgetting to verify activation, which would cause later installations to go to the wrong place. Checklists are most valuable for routine tasks that are easy to get mostly right but where a single missed step causes disproportionate trouble, and environment creation is exactly such a task.

A daily work checklist is shorter: activate the correct environment, verify it, work, record any new installations immediately, run tests after installations or updates, and deactivate when switching projects or finishing. That daily checklist, followed at the start and end of each session, prevents the most common daily mistakes, such as working in the wrong environment or forgetting to record a dependency. It also frames work sessions clearly, with activation at the start and deactivation at the end, which helps you switch contexts cleanly between projects. A checklist that is short enough to remember, or that is posted where you can see it, such as in a project readme or a personal note, is more likely to be followed than a long, detailed list that feels burdensome, so keep daily checklists brief and focused on the steps that matter most for correctness.

A sharing checklist, used before sharing a project, might include: update the requirements file, verify it by recreating an environment from it, run tests in the recreated environment, update setup instructions if anything changed, and confirm the environment folder is ignored in version control. Sharing checklists are particularly important because sharing is a moment when your work is judged by others, and a project that sets up cleanly on first try makes a strong impression, while one that fails due to a missing package or an unclear instruction undermines confidence, even if the code itself is good. A checklist for a standard-library-only project will be shorter than one for a project with many dependencies, and a checklist for a team project will include communication steps, such as notifying the team of dependency changes, that a solo project does not need. Review and update your checklists when you learn a new lesson, such as adding a verification step after encountering a problem that verification would have caught, so that checklists evolve with your experience. Checklists are living tools, not fixed rules, and their value lies in capturing lessons learned in a form that prevents repeating mistakes. Start with the checklists in this section, adapt them to your workflow, and let them guide your environment practice until the steps become automatic, at which point the checklist serves as a backup for moments when automatic habits fail, such as when you are tired or working on an unfamiliar project.

Check: Why is a checklist especially valuable for routine environment tasks where one missed step causes trouble?

Section 42: Environments on Different Operating Systems

Operating systems differ in how they handle paths, shells, scripts, and permissions, and those differences affect virtual environment commands, especially activation, which is shell-specific. On one system, activation may involve running a script for a particular shell, while on another, it may involve a different script or a different command syntax, and documentation that shows only one system's command can confuse users on another system. The concepts, however, are the same across systems: create an environment, activate it by adjusting the session's path, verify it, install packages into it, and deactivate when done. Learning the concepts, rather than memorising one system's commands, lets you adapt to any system, by finding the activation command for your shell and system, following its documentation, and verifying the result, rather than assuming a command from another system will work unchanged.

Path differences are another operating system consideration. Different systems use different separators in paths, different conventions for home folders, and different locations for system Python installations, all of which can affect how you refer to an environment folder or a Python executable. Using relative paths where possible, such as creating an environment inside your project folder and referring to it by a relative path from there, reduces system differences, because relative paths work similarly across systems, while absolute paths are system-specific. When you must use absolute paths, such as in documentation for a specific system, note that they are system-specific, and provide the equivalent for other systems where relevant, or describe the location in general terms, such as the project folder, rather than giving a path that only works on one system. Permissions, such as whether a script needs to be marked executable, also differ, with some systems requiring explicit executable permissions and others not. Those differences are usually handled automatically by environment creation tools, which create scripts with appropriate permissions for the system, but they can still cause issues when environments are copied between systems, which is another reason not to copy environments, but to recreate them on each system from requirements. Recreation on the target system ensures scripts, permissions, and paths are correct for that system, rather than inherited from another system where they may be wrong.

Finally, when collaborating across operating systems, test setup instructions on each system where the project will be used, or at least review them for system-specific assumptions, such as activation commands, path formats, or system dependencies that may differ. A setup instruction that works on one system but fails on another, due to an unnoticed system difference, will frustrate collaborators on the other system and may lead them to work around the problem in ways that diverge from the team's conventions. Cross-system awareness, achieved by learning concepts rather than commands, by using relative paths, by recreating environments per system, and by testing instructions where possible, makes projects more portable and teams more effective, regardless of which systems individual members use. Virtual environments are designed to be portable in concept, even though their folders are not portable in fact, and that distinction, recipe portable, folder not, is the key to working across systems successfully.

Check: Why should you recreate an environment on each operating system, rather than copying its folder between systems?

Section 43: Keeping pip and Tools Updated

Keeping pip and related tools updated is part of environment maintenance, because newer versions fix bugs, improve dependency resolution, and support newer package formats, while older versions may fail to install packages that rely on newer features, or may resolve dependencies less accurately, leading to conflicts that newer versions would handle correctly. Updating pip is usually done inside an environment, so that each environment's pip is current, though the pip used to create environments, which comes with the base Python, also matters, and updating it may require updating the base Python or using a separate mechanism, depending on the system. For most learners, updating pip within an active environment, when prompted or when installations behave unexpectedly, is sufficient, and it should be followed by a test that installations still work, to confirm the update did not break anything.

Tool updates should be done deliberately, not automatically for every session, because an update, like any change, can occasionally introduce a new bug or change behaviour in a way that affects your workflow. Update when there is a reason, such as a security fix, a feature you need, or a problem that a newer version is known to fix, and update one tool at a time, so that if a problem occurs, you know which update caused it. That deliberate approach, applied to pip and to other development tools, such as editors or version control tools, balances the benefit of staying current with the risk of disruption from untested updates. It also prevents the situation where many tools update at once, and a failure cannot be attributed to a specific update, making it harder to revert or to work around.

In addition to pip, other tools related to environments, such as tools for creating environments or for managing Python versions, also need occasional updates, and the same deliberate approach applies. Check for updates periodically, perhaps when you start a new project or when you encounter a problem that might be fixed by an update, and apply them in a controlled way, with a test afterwards. Document significant tool updates, especially if they change commands or behaviour, so that collaborators know what changed and can update their own tools accordingly, maintaining consistency across a team. Tool consistency, like environment consistency, reduces confusion, because everyone uses similar versions with similar behaviour, rather than some using old versions with different behaviour and others using new versions.

Finally, do not confuse updating tools with updating project dependencies. Updating pip does not update your project's packages, and updating packages does not update pip, though both are part of maintenance. Keep the two separate in your mind and in your workflow: update tools when tools need updating, update packages when packages need updating, and test after each kind of update, so that failures can be attributed correctly. That separation prevents the mistake of assuming a package problem is a tool problem, or vice versa, and it keeps maintenance organised, with each update verified independently. Regular, deliberate tool maintenance, combined with regular, deliberate dependency maintenance, keeps your development environment healthy, reliable, and ready for current projects, without the disruption of neglected tools suddenly needing a large, risky update all at once.

Check: Why should tool updates be done deliberately, one at a time, rather than all at once?

Section 44: Environment Size and Disk Space

Environment size and disk space become practical concerns as you accumulate projects, because each environment contains copies of its packages, and large packages, especially those for data, web, or scientific work, can make an environment hundreds of megabytes in size. Several such environments can quickly consume gigabytes, which may be significant on a computer with limited storage, or on a shared system where disk quotas apply. Understanding environment size helps you manage it: list environments by size periodically, identify the largest ones, and check whether they are still needed, or whether they contain packages that are no longer used and can be removed by recreating the environment from a clean requirements file. That periodic review, similar to cleaning up old environments, frees space and keeps your workspace manageable, without affecting current projects, provided their requirements are preserved.

Disk space management also involves pip's cache, which stores downloaded packages for reuse, and which can itself grow large over time, as it accumulates packages from many installations. The cache speeds up reinstallations, so deleting it entirely will make future installations slower, because packages must be downloaded again, but clearing it occasionally, when disk space is tight, is a reasonable trade-off, because packages can always be downloaded again from the index, provided network access is available. Check the cache size before clearing it, and consider whether the space saved is worth the slower future installations, rather than clearing it blindly. For most developers, the cache is not a major disk user compared with environments themselves, but it is worth knowing about, so that you can manage it if space becomes constrained.

Another disk consideration is that environments contain not only package files, but also copies or links to the Python interpreter and its standard library, though, as discussed earlier, environments typically link to the base Python rather than duplicating it fully, so their size is mostly packages, not Python itself. That linking design keeps environments relatively lightweight, but it also means that deleting the base Python can break environments, as discussed earlier, so disk management should not involve deleting Python installations to save space without checking which environments depend on them. Manage disk space by deleting old environments and clearing caches, not by deleting base Pythons that are still in use, because the latter causes breakage that is harder to repair than the space saved is worth.

Finally, consider disk space when choosing packages, especially large ones, for a project. A package that adds hundreds of megabytes for a small feature may not be worth its disk cost, especially on a limited device, compared with a smaller package or standard library code that achieves the same result with less space. That consideration is part of choosing packages wisely, as discussed earlier, where size is one factor among maintenance, documentation, and functionality. For most small projects, disk space is not a constraint, and package size need not dominate decisions, but for projects on limited devices, or for teams sharing limited storage, size awareness becomes important. Monitor environment sizes, clean up periodically, and choose packages with size in mind when it matters, so that disk space supports your work rather than constraining it unexpectedly.

Check: What contributes most to a virtual environment's disk size, and how can you reclaim space safely?

Section 45: Environments for Testing

Environments for testing deserve special attention, because tests should run in an environment that matches the project's runtime environment as closely as possible, so that test results reflect real behaviour, not an artefact of a different package set. If tests run in an environment with different package versions from the runtime environment, a test may pass in testing but fail in operation, or vice versa, because a package behaviour differs between versions. To prevent that, run tests in the project's main environment, or in a test environment created from the same requirements file, with the same versions, so that testing and runtime share the same dependency setup. That consistency is a basic requirement for trustworthy tests, and it is easy to achieve when environments are managed carefully, by using the same requirements for both purposes, or by creating a test environment from the same file.

Test environments may also need additional packages that are not needed at runtime, such as testing tools themselves, which should be recorded separately from runtime dependencies, so that deployment environments do not include them unnecessarily. That separation, between runtime and development or test dependencies, is discussed earlier for deployment, and it applies to testing as well: a test environment includes runtime packages plus test packages, while a runtime environment includes only runtime packages. Recording them separately, in different files or in different sections of a requirements file, makes that distinction clear, and it allows a test environment to be created by installing both sets, while a runtime environment installs only the runtime set. For small projects, the distinction may be informal, with test packages installed in the main environment and noted as such, but for larger projects, formal separation prevents deployment bloat and clarifies what is needed for each purpose.

Testing in a fresh environment, created from requirements, is a particularly strong verification, because it confirms that the project's tests pass not just in a long-lived development environment, which may contain unrecorded packages that mask missing dependencies, but in a clean environment built only from recorded files. Make fresh-environment testing part of your verification routine, alongside full test runs, to ensure both code and dependencies are correct.

Finally, test environments should be disposable, like other environments, so that they can be recreated freely for each test run, or for each major change, without concern for preserving state. That disposability encourages testing in clean conditions, rather than in a stale environment that may contain leftover state from previous runs, which could affect test results. For most small projects, recreating a test environment for every test run would be excessive, because it would slow testing considerably, but recreating it periodically, or when dependencies change, strikes a balance between cleanliness and speed. The key habit is to recognise when an environment's state might be affecting tests, such as when tests pass in one environment but fail in another, and to recreate a clean environment to check, rather than assuming the code is at fault. Environment-aware testing, where you consider the environment as part of the test setup, not as a fixed background, makes tests more reliable and debugging faster.

Check: Why should tests run in an environment matching the runtime environment, and why test occasionally in a fresh environment?

Section 46: A Project Setup Template

A project setup template is a standard sequence of steps and files for starting a new Python project, which you can reuse for each project, ensuring consistency and preventing missed steps. A basic template includes: a project folder with a clear name, a readme file with a description and setup instructions, a virtual environment with a conventional name, a requirements file, initially empty or with a note that the project uses only the standard library, an ignore file excluding the environment and other generated files, a main code folder or file, and a tests folder, even if initially empty, ready to hold tests as the project grows. That template, created for each new project, establishes good practices from the start, rather than retrofitting them later, when the project is already complex and habits are harder to change. Starting with a template is faster than starting from scratch each time, because the structure is already decided, and you can focus on the project's code, rather than on its organisation.

Customise the template to your needs and to the project's kind, rather than using it rigidly. A standard-library-only project, like the capstone, needs a simpler template, with a note about no external dependencies, while a web project needs a more complex template, with additional configuration and dependency files. A personal project needs a simpler readme than a team project, which needs more detailed setup and contribution instructions. The template is a starting point, not a straitjacket, and it should evolve as you learn what works for your projects, by adding steps or files that proved useful, and removing ones that proved unnecessary. That evolution, driven by experience, turns a generic template into a personal standard that reflects your workflow and your projects' needs, making each new project start smoother than the last.

Using a template also helps collaborators, because projects started from the same template have a familiar structure, making it easy for a collaborator to find code, tests, requirements, and documentation in the expected places, without needing project-specific guidance for basic navigation. That familiarity reduces onboarding time and confusion, especially in a team where many projects share a template, because a developer who knows one project's structure knows them all, at least at a high level. A template provides a good default, but each project has unique needs that may require deviations, such as a different folder structure for a particular kind of application, or additional configuration for a specific deployment target. Use the template as a checklist, not as a rule, and deviate deliberately when a project's needs justify it, documenting the deviation so that future readers understand why the project differs from the template. That balanced use, template as default, deviation as deliberate exception, gives you both consistency and flexibility, which is the goal of any standard: to make common cases easy and consistent, while still allowing uncommon cases to be handled thoughtfully. Start each project with your template, adapt it as needed, and let it evolve with your experience, and project setup will become a smooth, reliable routine rather than a source of uncertainty.

Check: What files and folders should a basic Python project setup template include, and why?

Section 47: Troubleshooting Installation Failures

Installation failures can have many causes, and troubleshooting them systematically, rather than retrying blindly, saves time and reveals the underlying problem. Start by reading the error message carefully, because pip's messages, though sometimes long, usually state the core problem: a package not found, a version conflict, a network error, an incompatible Python version, or a build failure for a source distribution. Identify which of those categories the message falls into, because each calls for a different response. A package not found may mean a misspelled name, a private package that requires a different index, or a package that does not exist for your Python version. A version conflict, discussed earlier, requires finding compatible versions. A network error may be transient, requiring a retry, or may indicate a proxy or firewall issue, requiring configuration. A build failure may indicate missing system tools, requiring installation of those tools or use of a pre-built distribution where available.

Next, verify the basics: is the correct environment active, is pip up to date enough for the package, is the Python version compatible with the package, and is there network access to the index? Those basic checks, done quickly, resolve many installation failures that seem complex but are actually simple environment or version mismatches. For example, an installation that fails because the package requires a newer Python than the environment's Python will succeed once the environment is recreated from a newer Python, while an installation that fails because the wrong environment is active will succeed once the correct environment is activated. Basic verification, before diving into detailed troubleshooting, prevents wasted effort on complex hypotheses when the cause is simple, and it is a habit that applies to debugging generally, not just to installations.

If basic checks do not resolve the failure, isolate the problem by trying a minimal installation: install the single package in a fresh, disposable environment, without other packages, to see whether it installs alone. If it does, the problem is likely a conflict with another package in the original environment, and you can add packages back one at a time to find the conflicting one, similar to halving in debugging. Isolation, by testing in a fresh environment, is a powerful troubleshooting technique for installations, because it separates package problems from environment problems, just as it separates function problems from caller problems in code debugging.

Finally, document installation failures and their solutions, especially for packages you use frequently, because the same failure may recur for collaborators or for future setups, and a documented solution saves repeated troubleshooting. Note the error message, the cause, and the fix, in a project note or a team guide, so that others can recognise the problem quickly if they encounter it. If a failure appears to be a genuine package bug, consider reporting it to the package's maintainers, with a minimal reproduction and the exact error message, following the guidance for asking for help effectively, because reporting helps the community and may lead to a fix that benefits everyone. Troubleshooting installations systematically, by reading messages, verifying basics, isolating, and documenting, turns a frustrating experience into a manageable, learnable one, and it builds expertise that makes future installations smoother.

Check: What systematic steps should you take when a pip installation fails?

Section 48: Environments and Security Revisited

Security in the context of environments involves both the safety of packages, discussed earlier, and the safety of environment management practices themselves, such as how environments are created, shared, and deleted. One practice is to avoid running code from untrusted environments, or installing packages from untrusted sources, as discussed, because code runs with your user's permissions regardless of environment, and an environment does not sandbox code. Another practice is to protect environment folders with appropriate permissions, so that other users on a shared computer cannot modify your environments' packages, potentially inserting malicious code that you would then run unknowingly. On a personal computer, this risk is low, but on a shared system, it matters, and setting permissions that restrict environment folders to your user is a sensible precaution, similar to protecting other personal files.

Another security consideration is credentials and configuration stored in environment-related files. Some projects store configuration, including sensitive values, in files inside the project folder, or in environment variables set during activation, and those files or variables can be accidentally shared if the project folder is copied or if activation scripts are shared. Avoid storing sensitive values in project files, in activation scripts, or in environment variables that are recorded in shell history, and use secure, approved methods for managing credentials, as emphasised throughout this book, rather than embedding them in files that might be shared. If a project requires a credential for development, document that it is required, and how to obtain it securely, without including the credential itself in documentation or in code. That discipline prevents accidental disclosure, which is a common security failure, where a credential is shared not through malice, but through carelessness in handling project files.

Environment deletion also has security implications. Deleting an environment folder removes its packages, but it does not necessarily securely erase files, depending on the system, and if an environment contained sensitive data, such as test data with personal information, or logs with sensitive values, simply deleting the folder may leave recoverable traces on disk. More commonly, the security issue with deletion is accidental deletion of the wrong folder, as discussed earlier, which is a reliability issue rather than a security issue, but which can still cause loss of code if version control is not used. Always ensure code is version-controlled before deleting an environment, so that code loss is not a consequence of environment cleanup.

Finally, keep environments updated, not just for functionality, but for security, because outdated packages may contain known vulnerabilities that have been fixed in newer versions, and an environment that is not updated for a long time accumulates such vulnerabilities. Regular updates, done safely with tests, as discussed earlier, are a security practice as well as a maintenance practice, because they reduce the window during which a known vulnerability remains in your project. Balance update frequency with stability, by prioritising security updates, testing them promptly, and scheduling regular reviews of dependencies for known issues, through release notes or security advisories for packages you use. Environment security, like package safety, is about awareness, caution, and regular maintenance, rather than about a single tool or setting, and it is a responsibility that grows with the sensitivity of the data and systems your projects handle.

Check: Why should credentials never be stored in activation scripts or project files that might be shared?

Section 49: From Environments to Collaboration

Environments are a foundation for collaboration, because they make it possible for multiple developers to work on the same project with equivalent setups, even on different computers, with different system configurations. That equivalence, achieved through shared requirements, shared Python version information, and shared setup instructions, rather than through shared environment folders, is what allows a team to develop together without constant environment-related failures. When a team member encounters a bug, others can reproduce it in their own equivalent environments, and when a fix is made, others can verify it in theirs, because the environments, though not identical in path, are equivalent in packages and versions. That reproducibility is the basis for effective collaboration on code, because it removes environment as a variable in debugging and testing, letting the team focus on code itself.

Collaboration also involves agreeing on environment practices, as discussed earlier, so that all team members create, activate, update, and share environments consistently. Those agreements, documented in a team guide or in project readmes, reduce friction when new members join, because they do not need to invent their own practices, and they reduce confusion when members help each other, because everyone's setup follows the same pattern. A team that agrees on folder names, requirements file names, Python versions, and update processes will spend less time troubleshooting environment differences and more time developing features, because environment differences, which are a common source of team friction, are minimised by convention. Conventions are not restrictive; they are enabling, because they free mental energy for the work that matters, rather than for setup decisions that should be standard.

As teams grow, environment practices may become more formal, with automated checks that verify a project's environment can be recreated, or that its requirements are up to date, or that its dependencies have no known security issues. Those automations, while beyond the scope of a first project, build on the same principles from this chapter: isolation, recording, recreation, and verification, applied automatically rather than manually. Learning those principles manually, as you have in this chapter, prepares you to understand and to use automated tools when you encounter them, because you will recognise what they are checking and why it matters, rather than treating them as magic. Manual practice is the foundation for automated practice, and it ensures you can diagnose problems when automation fails, because you understand the underlying process it automates.

Finally, collaboration through environments teaches a broader lesson about professional development: good practices are not just about writing code, but about making code shareable, reproducible, and maintainable by others. A project that only works on its author's computer is not truly finished, because it cannot be reviewed, tested, or deployed by anyone else, and its value is limited to its author. A project that can be set up by anyone, following clear instructions, with an equivalent environment, is a project that can grow, be maintained, and be trusted, because its setup is transparent and repeatable. That shareability, achieved through environments, requirements, documentation, and verification, is a hallmark of professional work, and it is a skill that will serve you in every team and every project you join, regardless of domain or scale.

Check: How do equivalent, recreatable environments enable effective team collaboration on code?

Section 50: Environment Habits to Keep

The habits from this chapter, taken together, form a simple, reliable approach to Python environments that will serve you throughout your development career, regardless of how complex your projects become. Create a separate environment for each project, activate it when you work, verify it before installing or running, install only packages you need and trust, record each installation immediately in a requirements file with versions, test after installations and updates, deactivate when you switch projects, exclude the environment folder from version control, and periodically recreate the environment from its requirements to verify reproducibility. That sequence, repeated consistently, prevents most environment problems, and when problems do occur, it gives you a clear framework for diagnosing them, by asking which step was missed or which record is out of date. Habits, practised until automatic, are more reliable than memory, especially under pressure, and environment habits are no exception.

Keep those habits proportionate to your projects' needs. A small, personal, standard-library-only project needs a simple version of the workflow, with an environment and a note about no external dependencies, while a large, shared, deployed project needs the full workflow, with frozen files, separated development dependencies, and verified recreation. Adjusting rigour to scale, rather than applying the same heavy process to every project or the same light process to every project, is a sign of judgement, and it prevents both over-engineering, which wastes time, and under-engineering, which causes failures. Review your habits periodically, as discussed earlier, and adjust them as your projects grow, your teams change, and your tools evolve, so that your workflow remains appropriate, not frozen in the form it had when you first learned it.

Finally, remember that environments are a means, not an end. Their purpose is to let you develop, test, share, and deploy Python programs reliably, without version conflicts, without breaking system tools, and without irreproducibility. When environments feel burdensome, check whether you are overcomplicating them, by adding unnecessary rigour for a simple project, or underusing them, by skipping steps that would prevent problems. The goal is a workflow that feels natural, that you follow without resentment, and that gives you confidence that your projects will work for others as well as for you. That confidence, built on isolation, recording, and verification, is the real benefit of virtual environments, and it is worth the small overhead of creating and managing them, because it removes a whole category of problems from development, letting you focus on writing good code, which is, after all, the point of programming.

Keep practising these habits on every project, even small ones, because habits formed on simple work transfer directly to complex work, while habits neglected early are hard to adopt later under pressure. When you start a new project, pause for a moment and set up its environment properly before writing code, rather than promising to do it later, because later rarely arrives and the cost of retrofitting isolation is higher than the cost of starting clean. Share your setup with others clearly, and you will find that good environment habits make collaboration smoother, deployment less stressful, and debugging faster, because environment differences stop being a hidden variable in every problem you investigate.

Check: What is the core sequence of environment habits that prevents most environment problems?

Practical Task

Create a new project folder with a virtual environment, activate it, install one well-known package you have evaluated, record it in requirements.txt with its version, verify the setup by listing installed packages, deactivate, then delete the environment and recreate it from requirements.txt alone, confirming the project still runs.

Chapter Questions

1. Why does each project need its own virtual environment, and what goes wrong without isolation?
2. Explain the difference between a requirements file and a frozen file, and when each is useful.
3. How do you resolve a dependency conflict, and why should you not force an incompatible installation?
4. Why is a virtual environment folder excluded from version control while requirements.txt is included?
5. Describe a safe process for updating a dependency, including how you verify the update.