Chapter 17 — Decorators and Context Managers
Section 1
A decorator is a callable that takes another callable, usually a function, and returns a new callable that extends or modifies the original behaviour without changing its source code. That description sounds abstract, but the practical idea is simple. When many functions need the same behaviour, such as logging their calls, timing their execution, checking their arguments, or caching their results, you can write that behaviour once as a decorator and apply it wherever it is needed. Each decorated function then gains the shared behaviour automatically, while its own code stays focused on its own job. This reuse is a powerful form of abstraction, because shared behaviour lives in one place where it can be tested, documented, and changed once to affect every decorated function at the same time.
The simplest decorator is a function that defines a wrapper function inside itself. The wrapper calls the original function, does something before or after that call, such as printing a message or recording how long the call took, and returns the original result to the caller. The decorator itself returns the wrapper, and that wrapper replaces the original function under its original name. From then on, calling the original name actually calls the wrapper, which in turn calls the original function at the right point in its own execution. Nothing inside the original function has changed. Only the binding of its name has changed, from the original function object to the wrapper object returned by the decorator.
That replacement is the key to understanding decorators. Decorating a function does not rewrite its code, move its statements, or alter the values it computes on its own. It changes what its name refers to after decoration, and that change lasts for as long as the name keeps pointing at the wrapper. Every call made through that name, from any part of the program, goes through the decorated behaviour. If you want the undecorated behaviour back, you must rebind the name to the original function explicitly, which is rarely done in ordinary code. In practice, therefore, decoration should be treated as a lasting decision about how a function behaves, not as a temporary effect that applies to one call only.
A small example makes the pattern concrete. The decorator below wraps any function and converts its result to upper case, leaving the original function free to return ordinary text. The wrapper accepts any arguments, so it works for functions with different signatures, and it passes those arguments straight through to the original function.
def shout(func):
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
return str(result).upper()
return wrapper
@shout
def greet(name):
return f"hello {name}"
print(greet("ada"))Decorators are worth learning because they turn repeated boilerplate into a named, reusable idea. Instead of copying logging or timing code into every function that needs it, you give that idea a name, implement it once, and apply it with a single line. That shared-behaviour idea is the thread running through this whole chapter: decorators and context managers both take behaviour that would otherwise be repeated and give it a single, testable home.
Check: In what sense does decorating a function change its name binding, rather than changing its source code?
Section 2
Decorator syntax using the @ symbol above a function definition is syntactic sugar for a simple assignment that happens immediately after the function is defined. Writing @decorator above def func(): is exactly equivalent to writing func = decorator(func) immediately after the function definition. Understanding that equivalence removes much of the mystery from decorators. There is no magic in the @ sign itself. It is a concise way to write a common pattern: define a function, then immediately replace it with a decorated version of itself returned by a decorator function. Once you see the assignment behind the syntax, decorators become ordinary Python code that you can reason about, test, and debug like any other code.
That equivalence also explains why decorators are applied at definition time rather than at call time. The assignment func = decorator(func) happens once, when the function is defined during module import or class definition, not on every call to that function. This distinction matters for how you write decorators. Code in the decorator body but outside its wrapper runs once at definition time, when the decorated function is created. Code inside the wrapper runs on every call to the decorated function. Setup work such as registering a function in a registry or validating its signature therefore belongs outside the wrapper, where it runs once. A common beginner mistake is to place such setup code inside the wrapper, where it needlessly runs on every call and can even produce different results if it depends on changing state.
Multiple decorators can be stacked above a single function definition by writing them one above another, from top to bottom. They are applied from bottom to top, starting with the decorator closest to the function definition and ending with the one furthest from it. Stacking @decorator1 above @decorator2 above a function is therefore equivalent to the nested assignment func = decorator1(decorator2(func)). The inner decorator wraps the original function first, and the outer decorator then wraps that result. At call time, the outermost wrapper runs first on entry and last on exit, like nested boxes. That order matters whenever wrappers interact, for example when a logging decorator and a timing decorator are combined: the order decides whether timing includes the time spent logging, or whether logging records the timed result.
A short example shows stacking in action. Each application of the decorator below doubles the result, so applying it twice multiplies the original result by four. Reading the stack from the bottom up tells you the order in which the wrappers were built, and reading a call from the outside in tells you the order in which they run.
def double(func):
def wrapper(*args, **kwargs):
return func(*args, **kwargs) * 2
return wrapper
@double
@double
def number():
return 3
print(number())When you stack decorators, always read them in both directions before you predict behaviour: bottom to top for construction, and top to bottom for execution. That habit prevents most stacking surprises.
Check: Why are stacked decorators applied bottom-to-top, and what does that mean for the order of their wrapper behaviour?
Section 3
Decorators with arguments are decorators that accept parameters of their own, such as a logging decorator that accepts a log level or a retry decorator that accepts a maximum number of attempts. They are implemented as decorator factories: functions that accept the decorator's arguments and return the actual decorator function, which in turn accepts the function to be decorated and returns its wrapper. That three-level structure is necessary because decorator syntax with arguments works in two steps. Writing @retry(times=3) first calls the factory retry(times=3) to obtain a decorator, and then applies that decorator to the function defined below it. The whole expression is equivalent to writing func = retry(times=3)(func), where the first call produces a decorator and the second call applies it.
The three levels can be confusing at first, because each nested function has a different role and runs at a different time. The outermost function, the factory, accepts decorator arguments and returns a decorator. The middle function, the decorator, accepts the function to be decorated and returns a wrapper. The innermost function, the wrapper, accepts the decorated function's arguments and returns its result after performing the decorator's behaviour. The factory runs at definition time, when the decorator is applied to a function. The decorator also runs at definition time, when it receives the function to be decorated. Only the wrapper runs on every call to the decorated function. Keeping those timings straight is essential, because code placed at the wrong level will run at the wrong time or far too often.
Decorator factories are useful because they make decorators configurable and reusable. A single retry implementation can serve one function with three attempts and another function with five attempts, where the second function needs more retries because of its particular failure characteristics. Without factories, you would need a separate decorator function for each configuration, which would duplicate logic and invite inconsistency. Factories also give you a natural place to validate configuration early. Arguments should be checked when the factory is called, at definition time, so that a misconfigured decorator fails fast during module import rather than late, when the decorated function is first called and the faulty configuration is harder to trace.
The example below shows the pattern in miniature. The factory repeat takes a count, the decorator receives the function, and the wrapper calls that function the requested number of times, returning the final result. Notice how each level closes over the values it needs: the decorator closes over times, and the wrapper closes over both times and the original function.
def repeat(times):
def decorator(func):
def wrapper(*args, **kwargs):
result = None
for _ in range(times):
result = func(*args, **kwargs)
return result
return wrapper
return decorator
@repeat(3)
def hello():
print("hi")
hello()When you write your own decorator factory, name its levels clearly, validate its arguments at the factory level, and test it with at least two different configurations. Those habits turn a confusing three-level structure into a reliable, reusable tool.
Check: Why does a decorator with arguments require three nested functions, and when does each level run?
Section 4
The functools.wraps decorator is an essential tool for writing decorators correctly, because it preserves metadata from the original function on the wrapper function that replaces it. Without wraps, a decorated function's name would be "wrapper", its docstring would be the wrapper's docstring or None, and its module would be the module where the decorator was defined rather than the module where the original function was defined. That loss causes real problems. Documentation tools rely on names and docstrings to generate useful documentation. Debugging tools display function names in tracebacks and logs, and a traceback full of "wrapper" entries is much harder to read. Introspection code that inspects metadata to decide how to call or document a function would likewise be misled by a wrapper masquerading under the wrong identity.
Using wraps is simple. Inside a decorator, you decorate the wrapper function with @wraps(func), where func is the original function passed to the decorator. That copies attributes such as __name__, __doc__, __module__, and __dict__ from the original function to the wrapper. It also sets the wrapper's __wrapped__ attribute to the original function, which lets tools reach the original through the wrapper when they need to, for example to unwrap a decorated function or to inspect its true signature. Python's inspect module follows the __wrapped__ chain, so a wrapped function can still report its original signature to tools that ask for it. For these reasons, using wraps should become a habit for every decorator you write, regardless of its size or purpose.
Without wraps, stacked decorators compound the metadata loss. Each wrapper in the stack replaces the previous wrapper's metadata with its own generic metadata, so after several layers of decoration, a function's name, docstring, and signature are completely lost to documentation and debugging tools. Using wraps at each layer preserves metadata through the entire stack, because each wrapper copies metadata from the function it wraps, which already carries the original function's metadata through its own wraps decoration. That is why consistency matters: metadata preservation in one layer is broken if a single layer in the stack forgets wraps, since the chain of copies is only as strong as its weakest link.
The example below shows the difference in practice. The decorator logs each call, but thanks to wraps, the decorated function still reports its own name and docstring afterwards, exactly as a reader would expect.
from functools import wraps
def logged(func):
@wraps(func)
def wrapper(*args, **kwargs):
print(f"calling {func.__name__}")
return func(*args, **kwargs)
return wrapper
@logged
def add(a, b):
"""Add two numbers."""
return a + b
print(add.__name__, add.__doc__)Treat wraps as part of the definition of a correct decorator, not as an optional polish. A wrapper that hides the function it wraps makes every tool that reads code a little less trustworthy.
Check: What metadata does functools.wraps preserve, and why does its absence compound through stacked decorators?
Section 5
Decorating methods requires understanding how decorators interact with method binding. A method is a function defined inside a class body that becomes a method when accessed through an instance or through the class, as discussed in earlier chapters on classes and descriptors. Decorating a method with a plain function decorator works in the same way as decorating an ordinary function: the decorator receives the method function and returns a wrapper function, and that wrapper becomes the method when accessed through an instance. The wrapper receives self as its first argument when called through an instance, just as the original method would have done. It must therefore accept *args and **kwargs, or at least self plus the method's other parameters, so that it forwards arguments correctly without dropping or misordering them. A wrapper that forgets self will fail on the first call, which makes this an easy mistake to catch but still a common one.
Class decorators are a related concept. Here a decorator is applied to an entire class definition rather than to a single function or method within it. Writing @decorator above class MyClass: is equivalent to writing MyClass = decorator(MyClass) immediately after the class definition. A class decorator receives the class object itself and can return either the same class object after modifying it in place, for example by adding methods or attributes, or a new class object that replaces the original entirely. Class decorators are useful for registering classes in a registry for later lookup by name or type, for adding boilerplate methods to many classes that share a pattern, and for validating a class definition at definition time before it is used.
Built-in decorators such as @property, @classmethod, and @staticmethod are special cases of method decoration that use the descriptor protocol from the previous chapter to change how a method binds. The @property decorator turns a method into a data descriptor that computes a value when accessed through an instance. The @classmethod decorator turns a method into one that receives the class itself as its first argument rather than an instance. The @staticmethod decorator turns a method into a plain function that receives neither an instance nor a class as its first argument. Understanding those built-ins as descriptors explains why they behave differently from ordinary function decorators when accessed through instances or classes, even though all of them use the same @ syntax.
The example below is deliberately plain, so the binding it relies on is easy to see. Each call to bump passes the instance automatically, and the method updates state belonging to that instance alone. A decorator wrapped around bump would sit in exactly that path, receiving the same instance as its first argument.
class Counter:
def __init__(self):
self.count = 0
def bump(self):
self.count += 1
return self.count
c = Counter()
print(c.bump(), c.bump())When you decorate a method, always check three things: that the wrapper accepts self, that it forwards every argument unchanged unless it has a reason not to, and that it preserves the original metadata with functools.wraps. Those checks keep method decoration as reliable as function decoration.
Check: How does a plain function decorator handle self when applied to a method, and what does a class decorator receive?
Section 6
Context managers were introduced in earlier chapters through the __enter__ and __exit__ protocol, but they deserve a full treatment here alongside decorators, because the two mechanisms share a common purpose. Both manage setup and cleanup around a block of code or around a function call, and both centralise behaviour that would otherwise be repeated. A context manager defines __enter__, which is called when execution enters a with block, and __exit__, which is called when execution leaves that block. Leaving can happen normally, by completing the block's statements, or exceptionally, through an exception raised inside the block. The __exit__ method receives information about any exception that occurred, or None values if the block completed normally, and it can suppress that exception by returning a true value or let it propagate by returning a false value.
The most common use of context managers is resource management. A resource such as a file, a lock, or a database connection is acquired in __enter__ and released in __exit__, with the guarantee that release happens even if an exception occurs inside the with block. Manual setup and cleanup with try and finally can also provide that guarantee, but context managers provide it more cleanly by encapsulating the setup and cleanup logic in a reusable manager object instead of duplicating it at every use site. That encapsulation is similar to decorators in an important way: both mechanisms centralise cross-cutting behaviour such as setup, cleanup, logging, or timing in a single reusable component, rather than scattering it across many places in a program where it can drift out of sync.
Context managers can also be used for purposes beyond resource management, such as timing a block of code, redirecting output temporarily, or changing a setting and restoring it afterwards. Those uses follow the same pattern: setup in __enter__, cleanup in __exit__, with cleanup guaranteed even on exception. The pattern is so useful that Python provides the contextlib module, covered later in this chapter, for building context managers from functions without defining a class with __enter__ and __exit__ methods. Understanding the underlying protocol remains essential, however, because it explains how with statements work, why cleanup is guaranteed, and when a class-based context manager is clearer than a function-based one, for example when the manager needs to hold complex state across the block.
The timer below is a minimal class-based manager. Its __enter__ announces the start and returns the manager itself, and its __exit__ announces the stop and returns False, which means any exception inside the block propagates normally after cleanup has run.
class Timer:
def __enter__(self):
print("start")
return self
def __exit__(self, exc_type, exc, tb):
print("stop")
return False
with Timer():
print("work")Whenever you write code that acquires something and must release it, ask whether a context manager should own that pair of actions. If the answer is yes, the with statement will enforce the pairing for you, on every path out of the block.
Check: What guarantee does a context manager provide that manual setup and cleanup code must also provide, and why is encapsulation better?
Section 7
The contextlib.contextmanager decorator provides a convenient way to create a context manager from a generator function, without defining a class with __enter__ and __exit__ methods. A generator function decorated with @contextmanager yields exactly once, at the point where the with block's body should execute. Code before the yield acts as setup logic, equivalent to __enter__, and code after the yield acts as cleanup logic, equivalent to __exit__. That generator-based approach is often clearer than a class-based approach for simple managers whose setup and cleanup can be expressed as sequential code around a single yield point. For such simple managers, clarity matters more than the flexibility of a class, and the generator form should be preferred in real programs.
The generator form also handles exceptions in a specific way that you should understand before writing contextmanager functions. An exception raised inside the with block is thrown into the generator at its yield point, so code after the yield can catch that exception in a try block around the yield and decide what to do. It can suppress the exception by handling it and returning normally from the generator, or it can let the exception propagate by not catching it or by re-raising it after performing cleanup. The standard pattern is a try and finally block around the yield: the finally clause guarantees cleanup even on exception, while leaving the exception itself to propagate unless you deliberately handle it. A contextmanager function that yields more than once, or that does not yield at all, raises a RuntimeError when used, which signals a bug in the manager's definition rather than a problem in the calling code.
Choosing between a class-based manager and a contextmanager function should be based on complexity. A contextmanager function is clearer for a simple manager with a single setup and cleanup pair that can be expressed sequentially around one yield. A class-based manager is clearer for a complex manager that needs to hold state across the block in instance attributes accessible through the object returned by __enter__, or that needs to be reused across multiple with blocks with different configurations set at construction time. Both forms should be tested with normal exit and with exceptional exit, to verify that cleanup happens in both cases and that exceptions propagate or are suppressed exactly as documented.
The example below follows the standard pattern. The message before the yield is the setup, the message in the finally clause is the cleanup, and the block's own output appears between them.
from contextlib import contextmanager
@contextmanager
def managed():
print("enter")
try:
yield
finally:
print("exit")
with managed():
print("work")Use the generator form when one yield tells the whole story, and reach for a class when the manager needs state, reuse, or a richer interface. Either way, test the exceptional path, because that is where a manager earns its keep.
Check: How does code before and after the yield in a @contextmanager function correspond to __enter__ and __exit__?
Section 8
Multiple context managers can be used together in a single with statement by separating them with commas, as in with open("a") as a, open("b") as b:. The statement enters each manager in order from left to right, and exits them in reverse order, from right to left, when the block completes or when an exception occurs inside it. That reverse-order exit matters for managers that depend on each other, such as a file and a lock protecting that file, where the order of release should mirror the order of acquisition. The guarantee is provided automatically by the with statement, without needing manual ordering logic in the block's code, and it holds on the exceptional path just as it does on the normal path. A reader can therefore trust the exit order by reading the statement from right to left.
Nested with statements provide an alternative way to use multiple managers together, by placing one with block inside another. In terms of entry and exit order, nesting is equivalent to a single statement with multiple managers, but it is less concise and more deeply indented. Nested statements should be preferred when the inner manager's setup depends on a value produced by the outer manager's __enter__, for example opening a file whose name is determined by reading a configuration file opened in the outer manager. That dependency cannot be expressed in a single with statement with multiple managers, because all managers in such a statement are created before any of them are entered. When the inner resource depends on the outer one, nesting is not just a stylistic choice; it is the only correct structure.
The contextlib.ExitStack class provides a way to manage a variable number of context managers dynamically at runtime, rather than statically in a with statement with a fixed number of managers. An ExitStack can enter managers one by one through its enter_context() method, and it will exit them all in reverse order when the stack's own with block exits. That is useful for programs that need to open a variable number of files based on user input or configuration data known only at runtime, where a fixed with statement cannot list the managers in advance. ExitStack also handles exceptions during entry of later managers by exiting managers that have already been entered before the exception occurred, which mirrors the guarantee of a single with statement with multiple managers.
The example below opens two files through a stack. Both files are closed when the stack's block exits, even though the code never names a fixed pair of managers in a single statement, and the same code would work for any number of files entered in a loop.
from contextlib import ExitStack
with ExitStack() as stack:
a = stack.enter_context(open("a.txt", "w"))
b = stack.enter_context(open("b.txt", "w"))
a.write("hello")
b.write("world")
print("both closed")Choose the fixed with statement when you know the managers in advance, nest when one depends on another, and reach for ExitStack when the number or identity of managers is only known at runtime.
Check: When should you use ExitStack instead of a single with statement with a fixed list of managers?
Section 9
Decorators and context managers can be combined in useful ways, because both mechanisms centralise cross-cutting behaviour around code that they wrap or enclose. A decorator wraps a function call with behaviour before and after that call, while a context manager encloses a block of code with setup and cleanup behaviour before and after that block. Those two mechanisms can be used together in two natural ways: by applying a decorator to a function that uses a with statement inside its body, or by implementing a decorator using a context manager inside its wrapper function, so that the manager governs a resource for the duration of the decorated function's execution. That combination is common in real programs that need both function-level decoration and block-level resource management for the same operation, and it works because the two mechanisms operate at different levels without interfering with each other.
A concrete example is a timing decorator implemented using a timing context manager inside its wrapper. The wrapper enters the timing manager, calls the original function inside the manager's with block, and exits the manager when the function returns or raises an exception. That is a clean way to implement timing, because the timing logic lives in the manager, where it can be reused independently of any decorator for timing arbitrary blocks of code that are not function calls. The decorator itself then becomes a thin wrapper that applies the manager to a function call specifically, rather than duplicating timing logic inside the decorator. The separation makes both components easier to test in isolation: the manager can be tested with arbitrary blocks, and the decorator can be tested with simple functions whose timing behaviour is easy to verify.
Another combination is a decorator that retries a function on failure, where each attempt is wrapped in a context manager that acquires and releases a resource needed for that attempt. A database connection, for example, should be opened fresh for each retry attempt rather than reused across attempts after a failure that may have left it in an inconsistent state. Placing the manager inside the retry loop ensures that resources are cleaned up correctly on each attempt, whether it succeeds or fails, before the next attempt begins. That cleanup guarantee is essential for retry logic to be safe in real programs that deal with unreliable external resources, such as network services or database connections that may fail intermittently and require retrying with fresh resources on each attempt.
The example below keeps the two ideas separate in a simpler form: a timing decorator that measures a function call directly. If you later extract its timing logic into a reusable manager, the decorator becomes thinner and the manager becomes testable on its own, which is exactly the separation described above.
import time
from functools import wraps
def timed(func):
@wraps(func)
def wrapper(*args, **kwargs):
start = time.time()
result = func(*args, **kwargs)
print(f"{func.__name__} took {time.time() - start:.4f}s")
return result
return wrapper
@timed
def work():
return sum(range(100))
print(work())When a decorator and a manager solve overlapping problems, put the reusable logic in the manager and let the decorator apply it. That division keeps each component small, testable, and usable on its own.
Check: How can a decorator be implemented using a context manager inside its wrapper, and why does that separation help testing?
Section 10
Decorator pitfalls include stacking order confusion, metadata loss without wraps, and state sharing between decorated functions when a decorator holds mutable state in a closure variable shared across all calls. That sharing can cause subtle bugs. A decorator intended to cache results per function may instead share a single cache across multiple functions decorated with the same decorator instance. The bug arises because a decorator factory called once creates a single cache object, which is then shared by all wrappers created by that factory call, unless the factory is called separately for each function to create a separate cache per function. The distinction between per-decorator state and per-function state must therefore be understood when writing stateful decorators such as caching or counting decorators. The safest habit is to create state inside the decorator function, so that each decorated function gets its own state, rather than in the factory, where one state object would be shared.
Another pitfall is decorating functions that are called recursively. A decorator that logs or times each call will log or time every recursive call as well as the top-level call, which may produce far more output than expected for a recursive function with many recursive calls. That behaviour should be anticipated when decorating recursive functions, and either accepted as correct for the decorator's purpose or avoided by decorating only a non-recursive wrapper function that calls an undecorated recursive helper internally. That separation between a decorated entry point and undecorated recursion is a useful pattern for decorators whose behaviour should apply only once per top-level call rather than once per recursive call. Without it, logs and timings for recursive code quickly become unreadable, and counts may be misinterpreted as top-level calls.
A third pitfall is overusing decorators for behaviour that would be clearer as explicit code inside a function. Decorating a single function with a complex decorator that is used nowhere else in a program adds indirection without adding reuse, and makes the function harder to understand than writing the behaviour directly in its body would have been. Decorators should therefore be reserved for behaviour that is genuinely shared across multiple functions, or that is complex enough to deserve its own reusable component with its own tests and documentation. A decorator used only once for simple behaviour is often a sign of over-engineering that should be simplified by inlining that behaviour into the function itself.
The counting decorator below shows state kept in the right place. The counter lives on the wrapper for one function, so each decorated function gets its own count, and testing it with two functions would prove that the counts do not leak between them.
def count_calls(func):
def wrapper(*args, **kwargs):
wrapper.calls += 1
return func(*args, **kwargs)
wrapper.calls = 0
return wrapper
@count_calls
def hello():
return "hi"
hello()
hello()
print(hello.calls)Test stateful decorators with two different functions, to prove their state does not leak between them.
Check: Why can a stateful decorator accidentally share state between different decorated functions, and how do you avoid it?
Section 11
Caching is one of the most useful applications of decorators, because many functions compute the same result repeatedly for the same arguments, and storing those results for reuse can save significant time, especially when the computation is expensive, such as a recursive calculation, a database query, or a complex transformation of data, that takes noticeable time, or resources, to perform, on each call, itself. A caching decorator wraps a function with a dictionary, keyed by its arguments, that stores each computed result, after its first computation, and returns the stored result, on subsequent calls, with the same arguments, without recomputing it, from scratch, which is a clean way, to add memoisation, to a function, without changing its source code, or its callers, who continue to call it, normally, unaware that some calls, are served from a cache, rather than computed afresh, on each invocation, itself. Python's standard library provides a ready-made caching decorator, functools.lru_cache, which implements a least-recently-used cache, with a configurable maximum size, that discards the least recently used entries, when the cache fills up, to keep memory usage, bounded, rather than growing, without limit, as new argument combinations, are encountered, during a program's execution, itself. Using lru_cache is usually better, than writing your own caching decorator, from scratch, because it has been tested, optimised, and documented, for general use, and because it handles details, such as thread safety, and cache statistics, through its cache_info() method, which reports hits, misses, and current size, for a cached function, itself, in a way, that a hand-written cache, would need to implement, separately, for each function, that uses it, itself. There are limits to caching, that should be understood, before applying it, to a function, in a real program, itself. A function should only be cached, if it is pure, meaning that it returns the same result, for the same arguments, every time, without side effects, that callers depend on, for correctness, of their programs, themselves, because caching a function, with side effects, such as writing to a file, or updating a database, would skip those side effects, on cache hits, causing incorrect behaviour, for programs, that rely on those side effects, happening, on every call, to that function, itself. Caching also requires, that arguments, be hashable, because they are used, as dictionary keys, in the cache, itself, so functions, with unhashable arguments, such as lists, or dictionaries, cannot be cached, directly, without first converting, those arguments, into a hashable form, such as a tuple, or a frozenset, for caching purposes, itself.
from functools import lru_cache
@lru_cache(maxsize=128)
def fib(n):
if n < 2:
return n
return fib(n - 1) + fib(n - 2)
print(fib(20), fib.cache_info())A cached function should also document that it is cached, so callers know its results may be reused.
Check: When is caching a function with a decorator safe, and when would it change the function's observable behaviour?
Section 12
A retry decorator is a practical example of a decorator with arguments, that wraps a function, with logic, to retry it, a specified number, of times, when it raises, a specified exception, before giving up, and propagating, that exception, to the caller, itself, which is useful, for functions, that interact, with unreliable external resources, such as network services, or database connections, that may fail, intermittently, due to temporary conditions, such as network congestion, or brief service unavailability, itself, where retrying, the same operation, a short time later, often succeeds, without any change, to the operation, itself, or to its arguments, themselves. A retry decorator, centralises, that retry logic, in one place, where it can be tested, and configured, with different, attempt counts, and delays, for different functions, in the same program, itself, rather than duplicating, retry loops, inside every function, that needs, retry behaviour, for its external interactions, themselves. Designing a retry decorator, requires decisions, about which exceptions, should trigger, a retry, and which, should propagate, immediately, without retrying, itself, because retrying, an operation, that fails, due to a permanent error, such as invalid arguments, or missing credentials, is wasteful, and will never succeed, no matter how many times, it is retried, with the same arguments, themselves, while retrying, an operation, that fails, due to a temporary error, such as a timeout, or a brief connection failure, is sensible, because a later attempt, may succeed, once the temporary condition, has cleared, itself. A well-designed retry decorator, therefore, accepts, a tuple, of retryable exception types, as an argument, and retries, only for those types, propagating, all other exceptions, immediately, without retrying, them, at all, itself, which is a distinction, that should be documented, clearly, for users, of the decorator, so that they know, which failures, will be retried, and which, will not, be retried, in a given program, itself. Testing, a, retry, decorator, should, include, tests, for, success, on, the, first, attempt, where, no, retry, occurs, at, all, itself, success, after, a, specified, number, of, failures, where, retries, occur, exactly, as, configured, itself, and, failure, after, exhausting, all, attempts, where, the, final, exception, propagates, to, the, caller, itself, as, a, whole, and, where, those, tests, should, use, a, fake, function, that, fails, a, known, number, of, times, before, succeeding, or, that, fails, always, with, a, known, exception, type, itself, so, that, retry, behaviour, can, be, verified, precisely, without, depending, on, real, external, services, that, fail, unpredictably, during, testing, itself, as, a, whole.
import time
from functools import wraps
def retry(times):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
for attempt in range(times):
try:
return func(*args, **kwargs)
except ValueError:
if attempt == times - 1:
raise
time.sleep(0.01)
return wrapper
return decoratorLog retry attempts, so failures are visible.
Check: Why should a retry decorator retry only specific exception types, and what danger applies to non-idempotent operations?
Section 13
A validation decorator checks a function's arguments before calling it, and raises a clear error if those arguments do not meet specified requirements, such as being of the correct type, within an acceptable range, or satisfying a custom predicate. That is useful for functions whose callers might pass invalid arguments by mistake, because failing fast with a clear error at the point of the call is better than failing later, deep inside the function's logic, with a confusing error that does not clearly indicate that invalid arguments were the problem in the first place. Validation decorators also centralise argument checking in one place, where it can be tested and reused across multiple functions that share the same validation requirements, instead of duplicating similar checks inside every function.
Validation decorators should be used judiciously, because Python's philosophy generally favours duck typing over strict type checking. A function should accept any object that provides the methods or behaviour it needs, rather than requiring arguments of a specific type that might exclude valid objects that happen to be of a different type but nevertheless provide the required behaviour. Strict validation that checks types exactly can therefore break duck typing and make functions less flexible than they should be for generic use, especially in programs that rely on polymorphism through protocols rather than through inheritance from specific classes. Validation should therefore focus on properties that genuinely matter for correctness, such as a number being positive or a string being non-empty, rather than on exact types that might exclude valid inputs unnecessarily.
A better approach for many validation needs is to validate at the boundaries of a program, such as when reading user input or data from an external source, rather than validating every internal function call with a decorator. Boundary validation catches invalid data once, when it enters a program, while internal validation on every call adds overhead and complexity to every function without providing additional safety for data that has already been validated at the boundary. Internal validation decorators should therefore be reserved for functions where invalid arguments would cause serious harm if not caught early, such as functions that write to files or databases, where invalid data could corrupt stored information.
The example below checks a property that matters, not a type. It rejects non-positive values with a clear message, while accepting any numeric value that supports comparison, which keeps the decorated function flexible for different numeric inputs.
from functools import wraps
def positive(func):
@wraps(func)
def wrapper(value, *args, **kwargs):
if value <= 0:
raise ValueError("value must be positive")
return func(value, *args, **kwargs)
return wrapper
@positive
def double(value):
return value * 2
print(double(3))Before adding a validation decorator, ask whether the check protects correctness, whether it belongs at a boundary instead, and whether it rejects any input the function could legitimately handle. If any answer is doubtful, leave the check out.
Check: Why should validation decorators focus on properties that matter for correctness, rather than on exact argument types?
Section 14
Preserving a decorated function's signature is a subtle issue, because a wrapper function that accepts *args and **kwargs has a generic signature that hides the original function's specific parameters. Tools that inspect signatures, such as documentation generators, development environments, and type checkers, would then see only the generic signature instead of the real parameters. The functools.wraps decorator preserves a function's name and docstring, but it does not by itself rewrite the wrapper's signature in a way that every tool can read directly. What it does is set the __wrapped__ attribute, and Python's inspect module follows that chain when possible to report the original function's signature for a decorated function. Using wraps is therefore usually sufficient for signature preservation in practice, for tools that use inspect.signature().
There are cases where signature preservation matters more directly, such as when a decorated function is called with keyword arguments that must match its original parameters exactly, or when a framework inspects a function's parameters to determine how to call it with injected dependencies or configuration values. In those cases, a wrapper that accepts only *args and **kwargs and forwards them unchanged to the original function will still work correctly for calling purposes, even though its own signature is generic rather than specific. Forwarding unchanged is sufficient for most decorators that do not need to inspect or modify specific arguments by name in their wrapper logic, and it is the approach you should prefer whenever you can.
Decorators that need to inspect specific arguments by name face a harder problem. They must bind the wrapper's generic arguments to the original function's parameters using inspect.signature().bind() or a similar mechanism, to determine which argument corresponds to which parameter. That binding logic should be tested thoroughly with positional arguments, keyword arguments, default values, and variable arguments, to ensure it handles all calling conventions correctly without misassigning arguments in edge cases, such as when a caller passes some arguments positionally and others by keyword in the same call. That complexity is a reason to prefer simple decorators that forward arguments unchanged whenever possible, rather than decorators that inspect specific arguments by name.
The example below shows the usual, reassuring case. Thanks to wraps, inspect.signature follows the __wrapped__ chain and reports the original parameters, including the default value, even though the wrapper itself accepts only generic arguments.
import inspect
from functools import wraps
def logged(func):
@wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
@logged
def greet(name, greeting="hello"):
return f"{greeting} {name}"
print(inspect.signature(greet))That binding complexity is also why many decorators avoid inspecting arguments by name at all, and instead treat the decorated function as a black box, forwarding whatever it receives unchanged. A related trap is changing a parameter name in the original function without updating a decorator that refers to it by name, which silently breaks the binding even though every call still looks correct. Keeping decorators argument-blind avoids that coupling entirely.
Check: How does functools.wraps help tools recover a decorated function's original signature, and when is that not enough?
Section 15
The contextlib module provides several ready-made context managers beyond contextmanager itself, and they are worth learning before you write custom managers for the same jobs. The suppress() manager suppresses specified exception types within its with block. The redirect_stdout() and redirect_stderr() managers redirect standard output or standard error to a different stream for the duration of the block. The closing() manager ensures that an object with a close() method is closed when the block exits. Those ready-made managers should be used in preference to custom managers for the same purposes, because they have been tested and documented for general use, and because their names express their intent clearly to readers. A reader who sees with suppress(...) understands the intention at a glance, without needing to inspect a custom implementation.
The suppress() manager deserves particular attention, because it makes exception suppression explicit and specific in a way that a bare try and except: pass block does not. Writing with suppress(FileNotFoundError): clearly states that a missing file is an expected condition that should be ignored within that block. A bare except: pass block, by contrast, suppresses all exceptions, including programming errors such as TypeError or NameError that should never be suppressed silently. That specificity is a safety benefit: it limits the damage when suppression is genuinely needed for a specific exception type, and it leaves genuine bugs visible instead of hiding them behind a silent pass.
Suppression should still be used sparingly, because even specific suppression can hide errors if applied too broadly. A large block wrapped in suppress() may swallow the expected exception in a place you did not intend, making a failure look like success and leaving later code to operate on missing or incomplete data. Keep suppressed blocks small, name the exact exception types you expect, and add a comment explaining why the exception is expected rather than a bug. If you find yourself suppressing the same exception in many places, that is a sign to reconsider the design, perhaps by checking for the condition explicitly before attempting the operation.
The examples below show the three helpers in their natural roles. The first ignores a missing file, the second captures printed output into a buffer, and the third guarantees that a closeable object is closed, all without any custom manager code.
import io
from contextlib import suppress, redirect_stdout
with suppress(FileNotFoundError):
open("missing.txt").read()
buffer = io.StringIO()
with redirect_stdout(buffer):
print("captured")
print(repr(buffer.getvalue()))Reach for the standard managers first, keep suppression narrow and specific, and reserve custom managers for behaviour the standard library does not already provide. The closing() manager follows the same logic: if an object offers close() but no with support, wrapping it states the cleanup contract plainly, instead of relying on every caller to remember a manual close in a finally block.
Check: Why is suppress() safer than a bare except: pass block, and what should suppression still be limited to?
Section 16
Two more contextlib managers deserve attention, because they solve common problems that arise when working with resources and with code that may or may not need a context manager. The first is closing(), which ensures that an object with a close() method is closed when its with block exits. The second is nullcontext(), which provides a no-operation context manager that does nothing on entry or exit. Both are small tools, but each removes a recurring awkwardness from ordinary code, and both are preferable to hand-rolled equivalents that readers must inspect to trust.
The closing() manager is useful for objects that have a close() method but are not themselves context managers, because they do not define __enter__ and __exit__ methods. Certain network or file-like objects returned by libraries require explicit closing but do not support with statements directly. Wrapping such an object in closing() allows it to be used in a with statement with guaranteed cleanup on exit, including on the exceptional path. The manager simply calls the object's close() method when the block exits, which is exactly the guarantee a try and finally block would provide, expressed in a form that states its intent directly. If the object already supports with, use it directly; closing() is for the gap where a closeable object lacks manager support.
The nullcontext() manager solves a different problem: code that conditionally needs a context manager. A function might need to acquire a lock in some cases but not in others, depending on its arguments or on program state. Writing that conditional logic with separate with blocks for each case would duplicate the block's body in both branches, which invites divergence as the code evolves. Using nullcontext() as a stand-in for the real manager in the case where no management is needed allows a single with statement to handle both cases uniformly, without duplicating the body. The nullcontext() manager can also return a value on entry through its argument, so code that uses the value produced by __enter__ works unchanged in both branches.
The example below shows the conditional pattern. The same block runs whether or not a lock is required, and only the manager chosen beforehand differs, which keeps the protected logic written once.
import threading
from contextlib import nullcontext
lock = threading.Lock()
use_lock = False
manager = lock if use_lock else nullcontext()
with manager:
print("protected work")Use closing() to give closeable objects the with guarantee, and use nullcontext() to keep conditionally managed code written once. Both managers make the structure of the code match the structure of the problem. When you test code that uses them, check both branches of the conditional case and both exit paths of the closing case, so the guarantee is verified rather than assumed.
Check: When is nullcontext() useful, and what problem does it solve for conditionally managed code?
Section 17
Asynchronous context managers extend the context manager protocol to programs that use asynchronous programming with async and await syntax. An asynchronous manager defines __aenter__ and __aexit__ methods, analogous to __enter__ and __exit__ for synchronous managers, but designed to work with async with statements instead of ordinary with statements. The concepts are parallel: setup happens on entry, cleanup happens on exit, and cleanup is guaranteed even when an exception occurs inside the block. The difference is that acquiring or releasing an asynchronous resource may itself need to wait, for example while a network connection is opened or a database session is established, and the async with form allows that waiting to happen without blocking the rest of the program.
Understanding asynchronous managers at a high level helps you recognise them when you encounter them in libraries that provide asynchronous resources, such as database connections or network clients that must be acquired and released asynchronously rather than synchronously. When you see async with in library code or in examples, you can now read it as the same setup and cleanup pattern you already know, applied to resources that arrive over time from external sources. You do not need to master the asynchronous protocol to benefit from that recognition, and you should not attempt to use an asynchronous manager with an ordinary with statement, or a synchronous manager with async with, because the two protocols are distinct even though their ideas match.
Asynchronous managers will be covered in detail in a later chapter devoted to asynchronous programming in Python, where their syntax, semantics, and use cases will be explained fully, with examples showing how they are defined and consumed in real programs that deal with asynchronous resources. They are not required knowledge for this chapter's core topics, which focus on synchronous decorators and synchronous context managers. The key point for now is simply to recognise that asynchronous managers exist as a parallel protocol, with similar guarantees and a similar shape, so that encountering them later feels like meeting a familiar idea in a new form rather than an entirely new concept.
The small illustration below is deliberately schematic, showing only the shape of the protocol. It is not a complete asynchronous program, and it is included so that the method names and the async with statement look familiar when you meet them in real library code.
class AsyncResource:
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
return False
async def main():
async with AsyncResource():
passFor now, remember the parallel: __aenter__ and __aexit__ play the roles of __enter__ and __exit__, and async with is the statement that drives them. The detailed treatment belongs to the asynchronous programming chapter, where it can be given the space it deserves.
Check: How do asynchronous context managers relate to synchronous ones, and why are they only introduced at a high level here?
Section 18
Class decorators that register classes in a registry provide a practical example of class decoration that appears frequently in real programs, such as plugin systems. In such a system, new functionality is added by defining a new class and decorating it with a registration decorator, which adds the class to a central registry mapping names to classes. That registry can then be used to create instances of registered classes by name, without needing to import each class explicitly at the point where it is needed. The pattern is useful for extensible programs that need to support new types of objects without modifying their core code to handle each new type explicitly. Adding a plugin becomes a matter of defining a class and decorating it, while the core program discovers available plugins through the registry.
Registration decorators should return the original class unchanged after registering it, so that the decorated class remains usable normally under its original name in the module where it was defined. That rule illustrates a general principle of class decorators: they should either return the original class after modifying it in place, or return a new class that is a clear replacement for the original, with documentation explaining what changed and why. Returning a new class without documentation can confuse users who expect the decorated class to be the same class they defined, with the same identity for identity checks, subclassing, and type checks. For registration decorators, whose purpose is to record or annotate a class rather than to replace it, in-place modification followed by returning the original class is the honest and least surprising choice.
Testing registration is straightforward and worth doing explicitly. A test should define a class with the decorator, check that the registry contains the class under the expected name, and check that the decorated name still refers to the original class object. It should also check that creating an instance through the registry produces an object of the expected type, and that an unregistered name is handled sensibly, for example by raising a clear error rather than returning a misleading default. Those tests protect the plugin contract, which is the whole point of the registry: new classes must be discoverable, and discovery must return exactly the class that was defined.
The example below implements the pattern in miniature. The decorator records each class in a dictionary keyed by its name, then returns the class untouched, so normal use of the class is unaffected by its registration.
REGISTRY = {}
def register(cls):
REGISTRY[cls.__name__] = cls
return cls
@register
class Widget:
pass
print(REGISTRY["Widget"] is Widget)Prefer registration decorators that record and return, rather than replace. A registry should make classes easier to find, not change what those classes are.
Check: Why should a registration class decorator return the original class unchanged, rather than a replacement class?
Section 19
Combining decorators in different orders can change a function's behaviour in ways that should be understood when stacking decorators in real programs. The order of decorators determines the order in which their wrappers execute around a function call. The outermost decorator, furthest from the function definition, executes first on entry to the decorated function and last on exit from it, which is analogous to nested function calls where the outermost call wraps all inner calls within its own execution. That ordering matters for decorators whose behaviour depends on what happens inside their wrapper, such as a timing decorator that measures time including the time taken by inner decorators if it is outermost, or excluding that time if it is innermost. The same function can therefore report different durations depending solely on where its timing decorator sits in the stack.
A concrete example is combining a logging decorator and a caching decorator. Placing logging outside caching means that every call is logged, including cache hits that do not recompute the function's result. Placing caching outside logging means that only cache misses, which actually compute a result, are logged, while cache hits return silently without logging at all. Neither order is inherently correct in all cases. The correct order depends on what the program needs to log: whether it needs a log of every call, including cache hits, for auditing purposes, or only a log of actual computations, for performance analysis. That decision should be made deliberately, based on the program's logging requirements, rather than by accident through stacking decorators in an arbitrary order.
The same reasoning applies to other combinations. A retry decorator outside a caching decorator retries the whole cached call, while a caching decorator outside a retry decorator may serve a cached failure or success without retrying at all, depending on what is cached. An authentication decorator should usually sit outside the behaviour it protects, so that unauthorised calls never reach inner wrappers or the function itself. When you stack decorators, draw the nesting in your head or on paper: the topmost decorator is the outermost box, and each call passes through the boxes from the outside in, then returns through them from the inside out.
The example below makes the ordering visible with two simple decorators. The first doubles a result and the second adds one, and swapping their order changes the final value, which shows why stacking order is a semantic choice rather than a cosmetic one.
def double(func):
def wrapper(*args, **kwargs):
return func(*args, **kwargs) * 2
return wrapper
def add_one(func):
def wrapper(*args, **kwargs):
return func(*args, **kwargs) + 1
return wrapper
@double
@add_one
def value():
return 3
print(value())Before stacking decorators, state in plain language what each layer should observe: every call, only computations, only authorised calls, or only the final result. Then place the layers so that the outermost wrapper is the one that must see the most, and the innermost is the one that must see the least.
Check: How does decorator stacking order affect whether a logging decorator records cache hits, when combined with a caching decorator?
Section 20
The first twenty sections of this chapter have covered decorator basics, decorator syntax, decorators with arguments, functools.wraps, decorating methods and classes, context manager basics, contextlib.contextmanager, multiple managers and ExitStack, combining decorators and context managers, decorator pitfalls, caching, retry, validation, signature preservation, contextlib helpers, closing and nullcontext, asynchronous managers at a high level, registration class decorators, and decorator stacking order. Each topic brought its own concepts and examples, but together they form a comprehensive toolkit for managing cross-cutting behaviour in Python programs. The remaining sections will develop those concepts further, with more advanced topics such as decorators for specific purposes like deprecation and access control, context managers for specific resources like locks and transactions, and design patterns for combining decorators and managers in larger programs that manage many resources and many cross-cutting concerns at once.
The principles that recur across all of these topics are worth naming explicitly, because they transfer to decorator and context manager tasks you have not yet met, and to new patterns you may design yourself. The first principle is centralisation: place shared behaviour in a single reusable component rather than scattering it across many places in a program. The second is reusability: design decorators and managers to be used across many functions or blocks without modification for each new use. The third is explicitness: make a decorator's or manager's behaviour clear from its name and documentation, so that users are not surprised by hidden side effects or by behaviour that differs from what its name suggests. Those three principles together form a practical framework for designing cross-cutting behaviour in any program.
Centralisation, reusability, and explicitness are not abstract slogans; they are checks you can apply to any new decorator or manager before you use it. Ask whether its behaviour lives in one place, or whether copies of the same logic are hiding elsewhere in the program. Ask whether it works for a second function or block without changes, or whether it secretly depends on the first use case. Ask whether a reader who sees only its name and docstring would predict its behaviour correctly, including its behaviour on exceptions and its effect on shared state. If any answer is unsatisfactory, the design needs revision before it is adopted more widely.
The short example below is a reminder that these principles apply even to small code. A class with a clear name and a single responsibility is easier to reason about than a clever construct whose purpose must be decoded, and the same holds for every decorator and manager you write.
class Book:
def __init__(self, title):
self.title = title
print(Book("Dune").title)Keep those three principles beside you as you work through the rest of this chapter, and test each new decorator or manager against them: is its behaviour centralised in one place, is it reusable across many functions or blocks without modification, and is its behaviour explicit from its name and documentation so that users are not surprised by hidden side effects when they use it in their own code? That framework will serve you well beyond this chapter whenever you design a new decorator, a new context manager, or a new combination of the two in a real program. Use it for every new cross-cutting behaviour you design.
Check: How do centralisation, reusability, and explicitness together form a framework for decorator and context manager design?
Section 21
A deprecation decorator marks a function as outdated and warns its callers that they should migrate to a newer alternative before the deprecated function is removed in a future version of a program or library. That warning is typically issued through Python's warnings module with a DeprecationWarning, which can be configured to show or hide depending on a program's warning settings. A deprecation decorator centralises warning logic in one place, so that every deprecated function in a program warns consistently, with the same message format and the same warning category, rather than duplicating warning calls inside every deprecated function. That consistency helps users of a library to recognise deprecation warnings and to act on them promptly, before the deprecated functions they use are removed in a later version.
A well-designed deprecation decorator should accept arguments such as the name of the replacement function to migrate to, and the version in which the deprecated function will be removed. Its warning message can then tell callers exactly what to do instead of calling the deprecated function, and when they must do it by, before their code breaks due to removal. That actionable information is far more useful than a vague warning that a function is deprecated without saying what to use instead or when it will disappear. A good message therefore contains three elements: the name of the deprecated function, the replacement to use, and the removal version or date. Without all three, callers must search documentation or source code to answer questions the warning itself should have answered.
Deprecation also needs a policy behind it, not just a decorator. A deprecated function should normally keep working for a stated transition period, so that callers have time to migrate without their programs breaking overnight. During that period, tests should verify both that the warning is issued and that the old function still behaves correctly, because a deprecation that silently changes behaviour is worse than no warning at all. When the removal version arrives, the function and its decorator should be removed together, and the change should be noted clearly in release notes. Keeping deprecated functions indefinitely, on the other hand, defeats the purpose of deprecation and leaves the library carrying code it intended to retire.
The example below shows the factory pattern applied to deprecation. The factory accepts a replacement name, the decorator applies wraps to preserve metadata, and the wrapper issues a warning before delegating to the original function, which continues to work as before.
import warnings
from functools import wraps
def deprecated(replacement):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
warnings.warn(
f"{func.__name__} is deprecated, use {replacement} instead",
DeprecationWarning,
stacklevel=2,
)
return func(*args, **kwargs)
return wrapper
return decorator
@deprecated("new_greet")
def old_greet(name):
return f"hello {name}"
print(old_greet("ada"))Write deprecation warnings as instructions, not as labels: what to use instead, and by when. Callers who receive that guidance can migrate calmly, and maintainers who give it earn the trust that makes future deprecations easier.
Check: What information should a deprecation warning include, so callers know what to do and by when?
Section 22
An access control decorator checks whether a caller is authorised to call a function before allowing that call to proceed. Authorisation might be based on a user's role in a local application, such as whether the current user is an administrator or a regular user in a program that manages its own users locally, without relying on external authentication services. The decorator raises a clear error, such as PermissionError, when a caller lacks the required authorisation for a given function, rather than allowing the call to proceed and then failing later with a confusing error deep inside the function's logic. That fail-fast behaviour is a security benefit for programs that protect sensitive operations such as deleting records or changing system settings, because it stops unauthorised work at the boundary instead of halfway through.
Access control decorators should be designed carefully, because they are security-critical components whose failure could allow unauthorised access to sensitive operations. Their logic should be simple, well tested, and reviewed carefully before deployment in any program that relies on them for security. Complexity in an access check is itself a risk: every branch is a chance to grant access in a case that should have been denied, and every implicit assumption is a chance for a caller to slip through. Keep the check to a single, clearly stated rule, such as requiring a named role, and keep the surrounding code free of side effects that could be exploited or misread.
Above all, access control logic should default to denying access when authorisation cannot be determined clearly for a given caller, rather than defaulting to allowing access in cases of uncertainty. A missing user, an unreadable role, or an ambiguous permission should all result in denial, with a clear error, until the ambiguity is resolved. Defaulting to allow in uncertain cases turns every gap in your data or logic into an open door, while defaulting to deny turns the same gap into a visible failure that can be investigated and fixed. That fail-closed principle applies to the decorator's handling of exceptions as well: if the authorisation check itself raises an unexpected error, the call should be denied rather than permitted on the grounds that no explicit denial was produced.
The example below illustrates the shape of the pattern with a deliberately simple role check. In a real program, the current user would come from a properly authenticated session, not from a global variable, but the structure of the check, the fail-closed default, and the clear error would remain the same.
from functools import wraps
current_user = {"name": "ada", "role": "user"}
def requires_admin(func):
@wraps(func)
def wrapper(*args, **kwargs):
if current_user.get("role") != "admin":
raise PermissionError("admin role required")
return func(*args, **kwargs)
return wrapper
@requires_admin
def delete_record(record_id):
return f"deleted {record_id}"
try:
delete_record(7)
except PermissionError as exc:
print(exc)Treat an access decorator as a security boundary, not as a convenience. Keep it simple, test the denial cases as thoroughly as the allowed cases, and never let uncertainty become permission.
Check: Why should an access control decorator default to denying access when authorisation cannot be determined?
Section 23
A timing decorator measures how long a function takes to execute and reports that duration through logging or printing. Timing information is useful for identifying slow functions in a program that need optimisation, and a timing decorator centralises timing logic in one place, so that any function can be timed by decorating it, without adding timing code inside every function that needs to be timed. That centralisation makes timing easy to add or remove during development and performance investigation, without modifying the functions being timed. It also keeps timing consistent: every timed function is measured the same way, with the same clock and the same reporting format, instead of each function inventing its own approach.
Timing decorators should use a monotonic clock, such as time.perf_counter(), rather than wall-clock time, such as time.time(), for measuring durations. Wall-clock time can jump forwards or backwards when a system's clock is adjusted during a program's execution, which would make measured durations incorrect for functions that execute across such an adjustment. A monotonic clock, by contrast, only moves forwards at a steady rate regardless of system clock adjustments, which makes it suitable for measuring elapsed time accurately in all cases. That distinction between monotonic and wall-clock time should be understood whenever you write timing code, because the wrong clock produces numbers that look precise while being quietly wrong.
A timing decorator must also record time even when the function raises. If timing code sits only on the success path, then the slowest and most interesting calls, the ones that fail after doing substantial work, disappear from your measurements entirely. The correct structure wraps the call in a try and finally block: record the start time, call the function, and in the finally clause compute and report the elapsed time before the exception propagates. That way, every call is timed, whether it returns normally or raises, and the exception itself is left untouched for the caller to handle. Timing should observe a function, never change its observable behaviour by swallowing its exceptions or altering its result.
The example below follows both rules. It uses perf_counter() for the measurement, reports in a finally clause so failures are timed too, and re-raises nothing because it never catches the exception in the first place.
import time
from functools import wraps
def timed(func):
@wraps(func)
def wrapper(*args, **kwargs):
start = time.perf_counter()
try:
return func(*args, **kwargs)
finally:
elapsed = time.perf_counter() - start
print(f"{func.__name__} took {elapsed:.4f}s")
return wrapper
@timed
def work():
return sum(range(1000))
print(work())Use timing as a diagnostic instrument, not as a permanent fixture on every function. Add it where you have a performance question, read what it tells you, and remove it when the question is answered. That discipline keeps timing data meaningful, instead of burying useful signals in a flood of routine measurements.
Check: Why should a timing decorator use a monotonic clock, and why must it record time even when the function raises?
Section 24
Locks as context managers provide a clean way to protect shared data in multi-threaded programs, where multiple threads might access or modify the same data concurrently. A lock object, such as threading.Lock(), can be used as a context manager in a with statement that acquires the lock on entry to the block and releases it on exit, whether normally or exceptionally. That guarantee is essential for preventing deadlocks in multi-threaded programs, because a lock that is not released after an exception would block all other threads waiting for that lock indefinitely. The critical section, the code that touches shared data, runs only while the lock is held, and no other thread holding the same lock can enter its own critical section at the same time.
Using locks as context managers is preferable to manual acquire and release calls, because manual calls require a try and finally block around the critical section to guarantee release on exception. That is more code and more error-prone than a simple with lock: statement, which provides the same guarantee automatically through the lock's context manager protocol. The preference for with statements over manual acquire and release should become a habit for all lock usage in multi-threaded programs, regardless of how simple the critical section appears at first glance, because even simple critical sections can raise exceptions unexpectedly. A single forgotten release, or a release placed on only one of several exit paths, is enough to hang a program in a way that is difficult to reproduce and diagnose.
Good lock discipline goes beyond using with. Keep critical sections short, so that threads do not wait longer than necessary, and do only the shared-data work inside them, moving unrelated computation outside. Avoid calling unknown code while holding a lock when you can, because that code might try to acquire another lock or block for a long time. If your program needs more than one lock, acquire them in a consistent order everywhere, to avoid the classic deadlock where two threads each hold one lock and wait for the other's. Those habits, together with the with guarantee, make threaded code substantially safer without making it substantially more complex.
The example below protects a shared counter. Each thread increments the counter inside the lock, so no increment is lost to a race between threads, and the lock is released even if an increment were to raise.
import threading
lock = threading.Lock()
counter = 0
def increment():
global counter
with lock:
counter += 1
threads = [threading.Thread(target=increment) for _ in range(10)]
for t in threads:
t.start()
for t in threads:
t.join()
print(counter)Whenever you reach for a lock, reach for with at the same time. The pairing is the point: acquisition without guaranteed release is a deadlock waiting for its first exception.
Check: Why is using a lock as a context manager safer than manual acquire and release calls?
Section 25
Transaction context managers provide a way to group multiple operations into a single atomic unit that either completes fully or rolls back entirely when any operation within it fails. That atomicity is essential for programs that update multiple related records together, such as transferring funds between accounts, where debiting one account without crediting another would leave data in an inconsistent state. A transaction manager implements atomicity by recording changes made inside its with block and committing those changes on normal exit, or discarding them on exceptional exit, when an exception occurs inside the block. That commit-or-rollback logic lives in the manager's __exit__ method, which receives exception information and decides whether to commit or roll back based on whether an exception occurred.
A simple in-memory transaction manager can be implemented by copying a dictionary on entry, allowing code inside the block to modify the copy freely, and then, on normal exit, replacing the original dictionary's contents with the copy's contents to commit the transaction, or, on exceptional exit, discarding the copy without modifying the original dictionary at all to roll back. That is a clear illustration of transaction semantics for teaching purposes, though real database transactions involve far more machinery, such as durable storage, concurrent access, and recovery after crashes. The teaching version is still valuable, because it makes the contract visible: inside the block, changes are provisional, and only a clean exit makes them permanent.
The decision in __exit__ deserves care. A normal exit, signalled by None exception arguments, means commit. An exceptional exit, signalled by an exception type and value, means roll back, and the manager should normally return a false value so the exception propagates to the caller after rollback, rather than swallowing evidence that something went wrong. Swallowing the exception would leave the caller believing the transaction succeeded, which is the opposite of what atomicity promises. The manager should also consider partial effects outside the guarded data, such as messages sent or files written inside the block, which a simple rollback cannot undo; those effects belong outside the transaction, or need their own compensation logic.
The example below shows the copy-and-commit idea directly. The successful block changes both balances, while the failing block changes neither, because its copy is discarded when the exception propagates.
accounts = {"ada": 100, "ben": 50}
class Transaction:
def __enter__(self):
self.copy = dict(accounts)
return self.copy
def __exit__(self, exc_type, exc, tb):
if exc_type is None:
accounts.clear()
accounts.update(self.copy)
return False
with Transaction() as data:
data["ada"] -= 30
data["ben"] += 30
print(accounts)Think of a transaction manager as a promise: either everything inside the block takes effect, or nothing does. The __exit__ method is where that promise is kept, and testing both outcomes is how you prove it holds.
Check: How does a transaction manager decide between committing and rolling back, based on its __exit__ arguments?
Section 26
A custom memoisation decorator built from scratch illustrates how caching decorators work internally, in a way that using functools.lru_cache alone does not reveal. A simple memoisation decorator holds a dictionary in its closure, keyed by a function's arguments, and checks that dictionary on each call. If the arguments are already present, it returns the stored result; otherwise it computes the result by calling the original function, stores it in the dictionary under its arguments as key, and returns it to the caller. That simple implementation reveals the core idea of memoisation clearly, without the additional complexity of least-recently-used eviction, cache statistics, or thread safety that lru_cache provides for production use. Building one by hand is therefore an excellent teaching exercise, even if you would not ship it.
That simple implementation also reveals limitations that a production cache must address. The first is unbounded growth: a cache dictionary grows without limit as new argument combinations are encountered, potentially consuming all available memory for a long-running program that processes many distinct argument combinations. That is why lru_cache imposes a maximum size and evicts least recently used entries when that size is exceeded, keeping memory bounded at the cost of occasionally recomputing an evicted result. The second limitation is that a simple cache keys directly on its arguments, so it requires hashable arguments and offers no control over which arguments form the key. Functions with unhashable arguments, such as lists, cannot be cached this way without converting their arguments into a hashable form first.
A third limitation is thread safety. A hand-written dictionary cache can misbehave if two threads call the decorated function with the same new arguments at the same time, potentially computing the result twice or observing a half-updated cache. The standard library cache handles concurrent access properly, which is one more reason to prefer it in threaded programs. A fourth limitation is observability: a simple cache provides no statistics about hits, misses, or current size, while lru_cache offers cache_info() for exactly that purpose, along with cache_clear() to empty the cache deliberately. Those tools matter when you need to verify that caching is actually helping, rather than assuming it is.
The example below is the teaching version, kept deliberately small so that the dictionary, the lookup, and the store are all visible. It works for pure functions with hashable arguments, and its limitations are the lesson.
from functools import wraps
def memoise(func):
cache = {}
@wraps(func)
def wrapper(*args):
if args not in cache:
cache[args] = func(*args)
return cache[args]
return wrapper
@memoise
def square(n):
return n * n
print(square(4), square(4))Build a simple cache once to understand the idea, then use lru_cache in real code. The standard tool's eviction, statistics, and thread safety are exactly the details a production cache needs and a teaching cache omits.
Check: What limitations does a simple hand-written memoisation decorator have, compared with functools.lru_cache?
Section 27
A context manager for temporarily changing a program's working directory illustrates how managers can encapsulate global state changes that must be restored after a block of code executes. Changing the working directory with os.chdir() affects all subsequent file operations in a program that use relative paths. Failing to restore the original directory after a block that needed a different directory temporarily would cause subsequent file operations elsewhere in the program to use the wrong directory for their relative paths. That is a bug that is hard to diagnose, because the file operations that fail are far from the code that changed the directory without restoring it. A context manager that saves the original directory on entry, changes to the new directory for the block, and restores the original directory on exit, whether normally or exceptionally, prevents that bug by construction rather than by careful manual restoration at every use site.
That pattern of saving state on entry, changing it for a block, and restoring it on exit applies to many kinds of global or shared state beyond working directories. Environment variables may need to be set temporarily for a block of code, logging levels may need to be changed temporarily for debugging, and configuration settings may need to be overridden temporarily for a specific operation. In each case, the risk is the same: a change made for a local purpose leaks into unrelated code if restoration is forgotten, skipped on an exceptional path, or placed where an early return bypasses it. A manager removes that risk by owning both halves of the change and guaranteeing the second half through the with protocol.
The implementation is short, which is part of its appeal. The manager records the current directory before changing it, yields control to the block, and restores the recorded directory in a finally clause so that restoration happens even when the block raises. Using contextlib.contextmanager makes the symmetry visible: the code before the yield is the setup, and the code in the finally clause is the cleanup. Tests should verify both exit paths, by checking the directory after a normal block and after a block that raises, because the exceptional path is where a manual implementation most often fails.
The example below follows that structure exactly. Inside the block, relative paths resolve against the temporary directory; after the block, they resolve against the original directory again, with no action required from the caller.
import os
from contextlib import contextmanager
@contextmanager
def working_directory(path):
previous = os.getcwd()
os.chdir(path)
try:
yield
finally:
os.chdir(previous)
print(os.getcwd())Whenever a change affects the whole program but is only needed for one block, wrap it in a manager. Global state should be borrowed, never left behind.
Check: Why is a context manager the right tool for temporarily changing global state such as the working directory?
Section 28
Testing decorators and context managers requires strategies that account for their wrapping behaviour. A decorated function cannot be tested in isolation from its decorator without either testing the decorator separately with a simple test function, or testing the decorated function together with its decorator as a combined unit. Both approaches are valuable for different purposes. Testing a decorator separately with a simple function that records its calls verifies the decorator's behaviour in isolation, without interference from the decorated function's own logic. Testing a decorated function together with its decorator verifies that the combination behaves correctly for that function's specific use case. Both kinds of tests together provide comprehensive coverage for decorated code, and neither kind alone is sufficient.
A recording stub is the key tool for testing a decorator in isolation. The stub is a tiny function that records every call it receives, including its arguments, and returns a predetermined value or raises a predetermined exception on demand. Decorating the stub and then calling it lets you observe exactly what the decorator does: whether it calls the function, how many times, with which arguments, in which order, and what it returns or raises. Because the stub has no real logic of its own, any behaviour you observe must come from the decorator, which makes failures easy to attribute. A stub that counts calls is especially useful for retry, caching, and rate limiting decorators, where the number of underlying calls is the behaviour under test.
Testing context managers similarly requires tests for normal exit, where cleanup happens as documented, for exceptional exit, where cleanup also happens and exceptions propagate or are suppressed exactly as documented, and for nested use, where multiple managers interact correctly without interfering with each other's cleanup logic. Those tests should use observable effects, such as files created or deleted, locks acquired or released, or state changed and restored, to verify manager behaviour directly rather than relying on internal implementation details. A manager test that only checks that no exception was raised has verified almost nothing; a test that checks the world after the block is the one that matters.
The example below shows a recording stub in its simplest form. It appends each call's arguments to a list, which a test can inspect afterwards to confirm how many times the function was called and with what values.
calls = []
def stub(value):
calls.append(value)
return value * 2
print(stub(3), calls)Test the decorator alone with a stub, test the manager on both exit paths with observable effects, and test the combination for the real use case. That three-layer approach catches wrapping bugs where they live, instead of letting them surface later as mysterious failures in production code. Keep this pattern in mind whenever you design a decorator.
Check: Why should decorator tests use a recording stub function, in addition to testing decorated functions together with their decorators?
Section 29
When decorators hurt readability is a topic that deserves attention, because decorators add indirection to code by inserting a wrapper function between a caller and the function it intends to call. That indirection can make code harder to understand when a reader must look up a decorator's definition to understand what a decorated function actually does when called, especially when multiple decorators are stacked above a single function, creating multiple layers of indirection that must all be understood together to understand a single function call. That readability cost should be weighed against the reuse benefit of decorators for each use case. A decorator that saves duplication across twenty functions earns its indirection; a decorator that decorates one function with behaviour used nowhere else often does not.
Decorators are most readable when their names clearly describe their behaviour, such as @timed, @cached, or @requires_admin, which tell a reader immediately what additional behaviour a decorated function will have when called. Decorators with vague or misleading names, such as @helper or @process, tell a reader nothing useful about what they do and force that reader to look up their definitions before understanding any decorated function that uses them. That naming principle applies to context managers as well as to decorators, because both mechanisms should be named for what they do rather than for how they are implemented internally. A manager called temporary_directory is self-explanatory; a manager called directory_helper is not, even if both do the same thing.
Good naming is necessary but not sufficient. Documentation matters too, because even a well-named decorator may have behaviour that its name cannot convey, such as whether it retries failures, how it handles exceptions, or whether it caches results. A short docstring on the decorator, stating its effect, its arguments, and its behaviour on failure, lets readers trust the name without reading the implementation. Stacking discipline matters as well: keep stacks short, order them deliberately, and comment any stack whose order is significant, so that a reader does not have to reconstruct the nesting from first principles.
The example below contrasts the two naming styles directly. Both decorators do the same thing, but only the first tells a reader what to expect before they read any further code.
def timed(func):
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
@timed
def fetch_users():
return []Name every decorator and manager for its observable effect, document its failure behaviour, and keep stacks short. Indirection that a reader can predict is a tool; indirection that a reader must decode is a cost. When a stack still feels hard to follow after naming and documentation, that is a signal to simplify the design itself, not to add another explanatory comment.
Check: How does good decorator naming reduce the readability cost of the indirection decorators introduce?
Section 30
The first thirty sections of this chapter have covered decorator basics, decorator syntax, decorators with arguments, functools.wraps, decorating methods and classes, context manager basics, contextlib.contextmanager, multiple managers and ExitStack, combining decorators and context managers, decorator pitfalls, caching, retry, validation, signature preservation, contextlib helpers, closing and nullcontext, asynchronous managers at a high level, registration class decorators, decorator stacking order, deprecation, access control, timing, locks, transactions, custom memoisation, working directory managers, testing strategies, and readability considerations. Each topic brought its own concepts and examples, but together they form a comprehensive toolkit for managing cross-cutting behaviour in Python programs. The remaining sections will develop those concepts further, with more advanced topics such as decorators for specific frameworks, context managers for specific resources, and design patterns for combining decorators and managers in larger programs that manage many resources and many cross-cutting concerns simultaneously.
The principles that recur across all of these topics are worth naming explicitly, because they transfer to decorator and context manager tasks you have not yet met, and to new patterns you may design yourself. Centralisation means placing shared behaviour in a single reusable component rather than scattering it across many places in a program. Reusability means designing decorators and managers to be used across many functions or blocks without modification for each new use. Explicitness means making a decorator's or manager's behaviour clear from its name and documentation, so that users are not surprised by hidden side effects or by behaviour that differs from what its name suggests. Those principles together form a practical framework for designing cross-cutting behaviour in any program, and they are the right lens for judging the advanced topics that follow.
Applying the framework to a new task starts with three questions. First, is the behaviour genuinely shared, so that centralising it saves duplication, or is it specific to one function, where explicit inline code would be clearer? Second, can the component be reused without modification, or does it secretly depend on details of its first use case? Third, would a reader predict its behaviour from its name and docstring alone, including its behaviour on exceptions? A component that passes all three checks is ready to be shared; a component that fails any of them needs revision before it spreads through a codebase.
The short example below is a reminder that these principles apply even to small code. A class with a clear name and a single responsibility is easier to reason about than a clever construct whose purpose must be decoded, and the same holds for every decorator and manager you write.
class Book:
def __init__(self, title):
self.title = title
print(Book("Dune").title)Keep those three principles beside you as you work through the final sections, and test each new decorator or manager against them: is its behaviour centralised in one place, is it reusable across many functions or blocks without modification, and is its behaviour explicit from its name and documentation so that users are not surprised by hidden side effects when they use it? That framework will serve you well beyond this chapter whenever you design a new decorator, a new context manager, or a new combination of the two in a real program. Use it for every new cross-cutting behaviour you design, and review it during code review.
Check: How do the principles from the first thirty sections help you design cross-cutting behaviour for new tasks?
Section 31
A logging decorator that accepts a log level as an argument illustrates how decorator factories can be used to create configurable logging behaviour for different functions in the same program. A function that handles sensitive operations might be logged at a detailed level for auditing purposes, while a function that performs routine operations might be logged at a less detailed level for general monitoring. That configurability through factory arguments allows a single logging decorator implementation to serve both purposes, without needing separate decorator functions for each log level. The factory takes the level, the decorator takes the function, and the wrapper logs each call at the configured level, which keeps the logging policy in one place while letting each function choose its own verbosity.
Logging decorators should use Python's logging module rather than print() calls for their output, because the logging module provides levels, formatting, and output destinations that can be configured centrally for an entire program, rather than scattering print statements throughout a program's code. With logging, an operator can raise or lower verbosity, redirect logs to a file, or add timestamps and module names, all without changing the decorated functions. With print(), none of that is possible without editing code, and log output mixes indistinguishably with ordinary program output. The logging module also lets libraries log without forcing output on their users, since a library's loggers can be silenced by default and enabled only when needed.
Logging decorators should log useful information about a function call, such as the function's name, its arguments in a form that does not reveal sensitive data, and its result or its exception if it raises one. Logging arguments requires care, because arguments might contain sensitive data, such as passwords, tokens, or personal information, that should not appear in log files that might be read by people who should not see that data. A logging decorator should therefore redact or summarise sensitive arguments, for example by logging argument names and types rather than values, or by allowing the caller to mark specific arguments as sensitive. It should also log failures, not just successes, because a log that omits exceptions gives a misleading picture of a program's health.
The example below uses the standard library logger and records both entry and outcome. It keeps the level configurable through the factory, preserves metadata with wraps, and leaves the original function's behaviour unchanged apart from its logging side effect.
import logging
from functools import wraps
logger = logging.getLogger(__name__)
def logged(level):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
logger.log(level, "calling %s", func.__name__)
result = func(*args, **kwargs)
logger.log(level, "%s returned", func.__name__)
return result
return wrapper
return decorator
@logged(logging.INFO)
def add(a, b):
return a + b
print(add(2, 3))Log through the logging module, choose levels deliberately per function, and treat arguments as potentially sensitive by default. A logging decorator that leaks secrets is worse than no logging at all.
Check: Why should a logging decorator use the logging module rather than print(), and what care is needed with arguments?
Section 32
A context manager for suppressing warnings temporarily illustrates how managers can encapsulate global state changes for warning filters. Python's warnings module allows programs to filter warnings by category or by message, and a program might need to suppress a specific warning category temporarily for a block of code that is known to trigger that warning for a legitimate reason, such as using a deprecated function intentionally during a migration period before its replacement is ready. Suppressing that warning globally for an entire program would hide other instances of the same warning elsewhere in the program that should not be suppressed, including new occurrences that signal genuine problems. A context manager that suppresses warnings only for its block provides a precise way to limit suppression to the specific code that needs it.
Python provides such a manager directly through warnings.catch_warnings(), which can be used as a context manager to temporarily modify warning filters within its block and then restore the original filters when the block exits. That standard manager should be used in preference to writing a custom warning-suppression manager from scratch, because it handles the save-and-restore logic correctly, including on the exceptional path, and because its behaviour is familiar to readers. Inside the block, you typically call warnings.simplefilter() to set the desired filter, for example to ignore a specific category, and the manager ensures that the previous filters return when the block ends. The standard manager is especially useful in tests that need to assert that a specific warning is issued by code under test, by capturing warnings within its block and examining them afterwards.
Limiting suppression in scope is not enough on its own; it must also be limited in category and justified in writing. Suppressing all warnings in a block, even a small one, hides unrelated warnings that may appear there in future, when the code changes. Suppressing a specific category for a specific reason, with a comment explaining the reason and the plan for removing the suppression, keeps the suppression honest and temporary. A suppression that survives a migration, long after its reason has disappeared, becomes a blind spot rather than a workaround, and should be treated as technical debt to be cleared.
The example below suppresses only DeprecationWarning inside a single block, using the standard manager. Warnings outside the block are unaffected, and the original filters are restored automatically when the block exits.
import warnings
with warnings.catch_warnings():
warnings.simplefilter("ignore", DeprecationWarning)
warnings.warn("old behaviour", DeprecationWarning)
print("suppressed block done")
print("filters restored")Suppress warnings narrowly, specifically, and temporarily. A block-scoped, category-specific suppression with a stated reason protects the code that needs it without blinding the rest of the program. Review each suppression when its reason expires, and remove it promptly rather than letting it linger unexamined.
Check: Why should warning suppression be limited to a specific block and category, rather than applied globally?
Section 33
A rate limiting decorator restricts how often a function can be called within a given time period. Rate limiting is useful for functions that interact with external services that impose limits on how many requests they will accept per second or per minute, or for functions that perform expensive operations that should not be repeated too frequently within a short time period. A rate limiting decorator centralises that limiting logic in one place, so that any function in a program can be rate limited by decorating it, without adding limiting code inside every function that needs to be limited. That centralisation makes rate limits easy to adjust centrally for an entire program, and it makes the limit visible at the function definition, where a reader can see that calls are throttled before reading the function's body.
A simple rate limiting decorator can be implemented by tracking the time of the last call to a decorated function in a closure variable, and on each new call checking whether enough time has passed since the last call to allow the new call to proceed, or whether the new call should wait until enough time has passed or raise an error indicating that the rate limit has been exceeded. That choice between waiting and raising an error should be documented for users of the decorator, so that they know what to expect when a decorated function is called too frequently. Waiting suits interactive or batch code that can afford a delay, while raising suits code that must fail fast and let the caller decide when to retry. Either choice is defensible, but a silent, undocumented choice is not, because callers cannot plan around behaviour they do not know about.
Rate limiting decorators should use a monotonic clock for their timing, for the same reason timing decorators do: a wall-clock adjustment must not grant extra calls or impose extra delays. State must also be kept per decorated function, not shared across functions, so that one function's limit does not throttle an unrelated function. Testing a rate limiter should not wait for real time. Instead, tests should inject a fake clock, a function that returns controllable time values, so that the decorator's decisions can be verified instantly: calls at the same fake time are limited, calls after the fake interval has advanced are allowed, and the waiting or raising behaviour matches the documentation.
The example below shows the structure with a minimum interval between calls. It records the last call time, computes how long to wait, and sleeps only when necessary, keeping the common case of a well-spaced call fast.
import time
from functools import wraps
def rate_limited(min_interval):
def decorator(func):
last = [0.0]
@wraps(func)
def wrapper(*args, **kwargs):
now = time.monotonic()
wait = min_interval - (now - last[0])
if wait > 0:
time.sleep(wait)
last[0] = time.monotonic()
return func(*args, **kwargs)
return wrapper
return decorator
@rate_limited(0.1)
def fetch(item):
return item
print(fetch("a"))Document whether excess calls wait or fail, use a monotonic clock, keep state per function, and test with a fake clock. A rate limiter that is predictable under test will be predictable in production too.
Check: Why should a rate limiting decorator use a monotonic clock, and how should its tests avoid waiting for real time?
Section 34
A context manager for temporary files illustrates how managers can encapsulate cleanup of files that are created temporarily for a block of code and that should be deleted when that block exits. Temporary files are useful for programs that need to write data to a file for processing by another program or library that expects file input rather than in-memory data. Those temporary files should be deleted after use, to avoid cluttering a system's temporary directory with leftover files from previous runs of a program, and to avoid leaking data that was only meant to exist briefly. A context manager that creates a temporary file on entry and deletes it on exit, whether normally or exceptionally, provides a clean way to guarantee that cleanup happens even if an exception occurs during processing of the temporary file.
Python provides such managers directly through the tempfile module, including TemporaryFile() and NamedTemporaryFile(), which can be used as context managers to create temporary files that are automatically deleted when their with blocks exit. Those standard tools should be used in preference to writing custom temporary file managers from scratch, because they handle platform differences, secure creation, and cleanup correctly. NamedTemporaryFile() is useful when a temporary file needs a name on disk for another program to open it by name, while TemporaryFile() is useful when a temporary file is needed only within the current program and never needs to be opened by name. Choosing between them is therefore a question of whether any other process or library needs the file's path.
Tests for temporary file handling should verify cleanup, not just successful processing. A test should check that the file exists inside the block, that its contents are correct, and that it no longer exists after the block exits normally. It should also check that the file no longer exists after a block that raises an exception, because the exceptional path is where a manual cleanup implementation most often fails. A test that only checks the happy path can pass for code that leaks files on every failure, which is precisely the failure a temporary file manager exists to prevent. Tests should also use unique temporary files rather than fixed paths, so that parallel or repeated test runs do not interfere with each other.
The example below uses the standard manager to write and read back a temporary file. The file is removed automatically when the block exits, and the program never needs to name or delete it explicitly.
import tempfile
with tempfile.NamedTemporaryFile("w+", delete=True) as tmp:
tmp.write("hello")
tmp.seek(0)
print(tmp.read())
print("temporary file cleaned up")Use the standard temporary file managers, choose the named form only when another consumer needs a path, and test cleanup on both exit paths. Temporary means temporary, and the manager is what makes that promise true.
Check: Why should temporary files be managed with a context manager, and what should tests verify about their cleanup?
Section 35
The singleton pattern, implemented through a decorator or through a class decorator, is a pattern that ensures that only one instance of a class is ever created in a program. That pattern is sometimes useful for classes that represent shared resources, such as configuration objects or connection pools, that should have a single shared instance throughout a program. It is often overused, however, in programs where a single instance is not actually required for correctness, but where a singleton is used out of habit or convenience rather than out of genuine necessity for shared state. Overuse can cause serious problems for testing, because a singleton's shared state persists across tests in a test suite, causing interference between tests, where one test's modifications to the singleton affect another test's expectations about its state.
A singleton decorator can be implemented by holding a single instance in a closure variable and returning that same instance on every call to the decorated class, rather than creating a new instance on each call. That simple implementation reveals the pattern's core idea clearly, without additional complexity for thread safety or for handling different constructor arguments on subsequent calls to the singleton class. Those additional complexities arise in real singleton implementations and are easy to get wrong: concurrent first calls may create two instances, and later calls with different arguments are silently ignored or, worse, applied inconsistently. A teaching implementation is therefore honest about what it omits, and a production implementation must address those gaps explicitly or not use the pattern at all.
The deeper objection to singletons is not implementation difficulty but design cost. A singleton is global state in disguise, and code that uses it depends on that global state implicitly, without declaring the dependency in its signature. That hidden dependency makes code harder to test, because a test cannot supply a fresh or fake instance without resetting global state, and harder to reuse, because the code cannot be run with two different configurations at once. Passing a shared instance explicitly, as an argument to the functions or objects that need it, keeps the sharing where it is needed while making the dependency visible. Callers can then share one instance in production and supply separate instances in tests, without any global reset.
The example below shows the mechanism in miniature, so the closure-held instance is visible. It is included to explain the pattern, not to recommend it as a default design.
def singleton(cls):
instance = None
def get_instance(*args, **kwargs):
nonlocal instance
if instance is None:
instance = cls(*args, **kwargs)
return instance
return get_instance
@singleton
class Config:
def __init__(self):
self.value = 1
print(Config() is Config())Before reaching for a singleton, ask whether sharing is truly required, and whether passing the instance explicitly would serve just as well. Explicit sharing gives you the same single instance in production, without the hidden global state that makes testing painful.
Check: Why are singletons often discouraged in favour of passing a shared instance explicitly, especially for testing?
Section 36
A context manager for redirecting input illustrates how managers can encapsulate changes to a program's input source for a block of code. A program that reads input from standard input through input() calls might need to be tested with predetermined input values, without requiring a human to type those values interactively during testing. A context manager that temporarily replaces standard input with a string buffer containing predetermined input values for the duration of its block provides a clean way to test interactive code without manual intervention. That manager should restore the original standard input when its block exits, whether normally or exceptionally, to avoid affecting subsequent code that expects standard input to be its normal interactive source.
Python provides tools for such redirection through the contextlib module and through mocking libraries used in testing. The contextlib.redirect_stdout family covers output, while for input the common approach is to replace sys.stdin temporarily or to use unittest.mock.patch() as a context manager. The patch() manager temporarily replaces a specified object, such as standard input or a function, with a mock object for the duration of its block, and then restores the original object when its block exits. That restoration guarantee is a standard technique for testing code that depends on external resources or on user input, without letting test doubles leak into other tests. Whichever tool you use, the contract is the same: the replacement lasts exactly for the block, and the original returns afterwards.
Testing interactive code this way requires care about what the predetermined input contains. Each input() call consumes one line, so the buffer must provide lines in the order the program reads them, including any prompts the program expects to display. A test that supplies too few lines will fail with an unexpected end of input, which is itself a useful signal that the program read more than the test anticipated. A test should also verify behaviour after the block, to confirm that standard input was restored, because a leaked replacement can make later tests behave strangely in ways that are hard to trace back to the redirecting test.
The example below redirects standard input to a buffer containing a single name, so that an input() call inside the block reads from the buffer instead of waiting for a person. Outside the block, standard input is restored.
import io
import sys
from contextlib import redirect_stdin
with redirect_stdin(io.StringIO("ada\n")):
name = input("Name: ")
print(f"hello {name}")Use a manager to feed predetermined input during tests, supply lines in the order they are read, and verify restoration after the block ends. Interactive code tested this way becomes repeatable, which is the point of automating its tests at all. Keep each test's input small and focused, so a failure points clearly at one behaviour.
Check: How does redirecting standard input inside a context manager help test interactive code, and what must be restored on exit?
Section 37
Building a decorator library for a program or for a team involves designing a collection of decorators that share common conventions for naming, arguments, and behaviour. Those conventions should be documented clearly for users of the library, so that they know how to use each decorator correctly without needing to read its implementation in detail for every use case. Useful conventions include using functools.wraps in every decorator in the library, accepting arguments through decorator factories consistently for all configurable decorators, and documenting each decorator's behaviour for normal calls, exceptional calls, and stacked use with other decorators in the library. Those conventions together make a decorator library feel coherent and predictable to its users, rather than feeling like a collection of unrelated decorators with inconsistent behaviour for similar tasks.
A decorator library should also be tested comprehensively, with tests for each decorator in isolation using simple test functions that record their calls, as well as tests for combinations of decorators that are commonly used together in real programs that use the library. Those combination tests should verify that decorators interact correctly when stacked in different orders, rather than assuming that decorators that work correctly in isolation will also work correctly together. Stacking is where subtle interactions live: a caching decorator may hide calls from a logging decorator, a retry decorator may multiply the effects of a counting decorator, and a validation decorator may reject arguments that another decorator has transformed. Only combination tests reveal those interactions before users encounter them.
Documentation for a decorator library should describe more than each decorator's purpose. For each decorator, it should state whether it preserves metadata, whether it is safe to stack with the library's other decorators and in which orders, how it behaves when the decorated function raises, and whether it holds state and, if so, whether that state is per function or shared. That information lets users combine decorators confidently instead of discovering interactions by trial and error. Examples should show the common stacks, not just single decorators, because the common stacks are where users will actually spend their time.
The example below is a tiny library-style pair: two decorators that both use wraps and both forward arguments unchanged, so that stacking them in either order remains predictable. Small, consistent building blocks are what make a larger library trustworthy.
from functools import wraps
def logged(func):
@wraps(func)
def wrapper(*args, **kwargs):
print(f"calling {func.__name__}")
return func(*args, **kwargs)
return wrapper
def timed(func):
@wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
@logged
@timed
def work():
return 1
print(work())Design a decorator library around shared conventions, document stacking and failure behaviour explicitly, and test combinations, not just individual decorators. Consistency across the library is what turns separate decorators into a tool users can rely on.
Check: Why should a decorator library test combinations of decorators, not just each decorator in isolation?
Section 38
The contextlib.ExitStack class provides advanced features beyond simply entering multiple managers dynamically, including the ability to register arbitrary cleanup callbacks through its callback() method. That method schedules a function to be called when the stack's with block exits, whether normally or exceptionally. Those callbacks are executed in reverse order of their registration, similar to how managers entered through enter_context() are exited in reverse order of their entry. That callback feature is useful for cleanup tasks that are not naturally expressed as context managers, such as deleting a temporary file created during a block, or restoring a global setting changed during a block, without needing to define a full context manager class or function for each such cleanup task.
ExitStack also provides a push() method for registering managers that have already been entered outside the stack, so that they will be exited by the stack when its block exits. That is useful for code that acquires resources before entering a stack's block and wants those resources to be released together with resources acquired inside the stack's block. The feature should be used carefully, because pushing a manager that is already managed elsewhere could cause it to be exited twice if both the stack and the original owner try to exit it. Double exit is usually harmless for idempotent cleanup, but it can be a real bug for managers whose exit has side effects that must happen exactly once. When in doubt, let one owner manage each resource, and use push() only when ownership is genuinely being transferred to the stack.
Callbacks and pushed managers share the stack's exit guarantees, which is what makes them safe to combine. If an exception occurs inside the block, callbacks still run, in reverse order, and managers still exit, in reverse order, with exceptions during cleanup handled according to the same rules as an ordinary with statement. That uniformity means you can mix managers, callbacks, and pushed managers in a single stack without learning a separate set of rules for each kind of cleanup. The stack becomes a single place where all of a block's cleanup obligations are registered, which is much easier to audit than cleanup scattered across finally clauses.
The example below registers a simple callback alongside a managed resource. The callback runs when the block exits, even though it was never a context manager itself, which shows why callbacks are a useful complement to managers.
from contextlib import ExitStack
done = []
with ExitStack() as stack:
stack.callback(done.append, "cleaned")
print("work")
print(done)Use callback() for cleanup that is a single function call rather than a manager, use push() only when ownership truly transfers, and rely on reverse-order exit to keep combined cleanup predictable.
Check: What cleanup tasks can ExitStack.callback() handle that are not naturally expressed as context managers?
Section 39
Decorators in frameworks provide a familiar example of decorator use at scale. Web frameworks use decorators to register functions as handlers for specific routes or URLs in a web application. Writing @app.route("/hello") above a function definition registers that function as the handler for requests to the "/hello" URL, in a central routing table maintained by the framework. That registration pattern is similar to the class registration pattern discussed earlier in this chapter, but applied to functions rather than to classes. It makes web applications easy to read, because each handler function's route is visible directly above its definition, rather than being specified in a separate routing configuration file that must be kept in sync with handler definitions manually. When a route and its handler live together, they cannot drift apart unnoticed.
Framework decorators also illustrate how decorators can be used for cross-cutting concerns in frameworks, such as authentication decorators that check whether a request's user is authorised to access a specific route before allowing the handler function for that route to execute, or caching decorators that cache responses for specific routes to avoid recomputing them on every request. Those framework decorators follow the same principles as ordinary decorators discussed in this chapter: they centralise shared behaviour, they preserve metadata where it matters, and their stacking order determines what they observe. An authentication decorator outside a caching decorator, for example, ensures that unauthorised requests never receive a cached response, while the reverse order could leak cached content to callers who should have been denied.
There is an important distinction between registration decorators and wrapper decorators in frameworks. A registration decorator, such as a route decorator, typically returns the original function unchanged after recording it in a table, because its job is to note the function, not to change how it behaves. A wrapper decorator, such as an authentication or timing decorator, returns a new wrapper function that adds behaviour around each call. Confusing the two kinds causes real bugs: wrapping a handler when you meant only to register it can break introspection, while registering a function when you meant to wrap it leaves the intended behaviour absent. Reading a framework decorator's documentation to learn which kind it is, before stacking it with others, prevents that confusion.
The miniature framework below shows registration without wrapping. The decorator records the function in a routing table and returns it unchanged, so the function remains directly callable and its identity is preserved.
ROUTES = {}
def route(path):
def decorator(func):
ROUTES[path] = func
return func
return decorator
@route("/hello")
def hello():
return "hello"
print(ROUTES["/hello"]())When you use framework decorators, distinguish registration from wrapping, check stacking order for security-sensitive layers, and keep routes beside their handlers. Frameworks scale decorators by making those distinctions explicit, and your own code should do the same.
Check: How does a framework route decorator differ from a wrapper decorator, in terms of what it returns?
Section 40
Sections thirty-one through forty of this chapter have covered logging decorators, warning suppression managers, rate limiting decorators, temporary file managers, singleton decorators, input redirection managers, decorator libraries, advanced ExitStack features, and framework decorators. Each topic built on the foundations laid in the first thirty sections, and together they extend the basic decorator and context manager toolkit into a more complete toolkit for real programs. Those programs range from web applications to data processing pipelines to system utilities that interact with files, locks, and other shared resources, and the extended toolkit addresses the cross-cutting behaviour and resource management those programs need in many different contexts. The remaining sections will complete that toolkit with final topics, such as decorators for specific purposes like memoisation with custom policies, context managers for specific resources like lock files, and design patterns for combining decorators and managers in larger programs that manage many resources and many cross-cutting concerns simultaneously.
The principles that recur across all of these topics are worth naming explicitly, because they transfer to decorator and context manager tasks you have not yet met, and to new patterns you may design yourself. Centralisation means placing shared behaviour in a single reusable component rather than scattering it across many places in a program. Reusability means designing decorators and managers to be used across many functions or blocks without modification for each new use. Explicitness means making a decorator's or manager's behaviour clear from its name and documentation, so that users are not surprised by hidden side effects or by behaviour that differs from what its name suggests. Those principles together form a practical framework for designing cross-cutting behaviour in any program, and they apply just as strongly to the advanced topics as to the basics.
The second half of the chapter also adds a caution that the first half only hinted at: powerful, reusable components carry responsibility. A logging decorator that records sensitive arguments, a singleton that hides global state, or a framework decorator that wraps when it should only register can each cause harm precisely because they are applied widely. Reuse multiplies both benefits and mistakes, so widely used components deserve more testing and clearer documentation than one-off code.
The short example below is a reminder that these principles apply even to small code. A class with a clear name and a single responsibility is easier to reason about than a clever construct whose purpose must be decoded, and the same holds for every decorator and manager you write.
class Book:
def __init__(self, title):
self.title = title
print(Book("Dune").title)Keep those three principles beside you as you work through the final sections, and test each new decorator or manager against them: is its behaviour centralised in one place, is it reusable across many functions or blocks without modification, and is its behaviour explicit from its name and documentation so that users are not surprised by hidden side effects when they use it? That framework will serve you well beyond this chapter whenever you design a new decorator, a new context manager, or a new combination of the two in a real program. Use it for every new cross-cutting behaviour you design.
Check: How do the topics in sections 31–40 extend the basic decorator and context manager toolkit from the first thirty sections?
Section 41
A memoisation decorator with a custom key function extends basic memoisation by allowing the decorator's user to specify how cache keys are derived from a function's arguments. That customisation is useful when a function's arguments are unhashable, such as lists or dictionaries, that cannot be used directly as dictionary keys in a cache, or when a function's arguments contain information that is irrelevant to its result, such as a logging flag that does not affect the computed value but that would otherwise create separate cache entries for calls that should share the same cached result. A custom key function can transform arguments into a hashable key that captures only the information relevant to a function's result, which makes caching possible in cases where a plain argument-based cache would fail or waste entries.
That customisation should be designed carefully, because a key function that ignores relevant argument information will cause incorrect cache hits, where calls with different relevant arguments return the same cached result incorrectly. Conversely, a key function that includes irrelevant argument information will cause unnecessary cache misses, where calls that should share a cached result instead compute it afresh wastefully. The balance between correctness and efficiency in key design should be tested thoroughly, with calls that differ only in irrelevant arguments, which should hit the cache, and calls that differ in relevant arguments, which should miss the cache and compute fresh results. Those two test cases, taken together, pin down exactly what the key function must include and what it must exclude, and they should be written before the key function is considered complete.
Designing a key function also requires thinking about how arguments are normalised. A list argument might be converted to a tuple, a dictionary argument might be converted to a sorted tuple of its items, and a numeric argument might be normalised so that equivalent values produce the same key. Normalisation must be consistent: two calls that should share a result must produce equal keys, and two calls that should differ must produce different keys. It must also be total, in the sense that every argument combination the function accepts can be converted to a key, or the decorator must raise a clear error for arguments it cannot key, rather than failing obscurely inside the cache lookup.
The example below caches by the first argument only, ignoring a debug flag that does not affect the result. Calls with the same value but different flags share one entry, which is correct only because the flag genuinely does not change the computed value.
from functools import wraps
def memoise_by(key_func):
def decorator(func):
cache = {}
@wraps(func)
def wrapper(*args, **kwargs):
key = key_func(*args, **kwargs)
if key not in cache:
cache[key] = func(*args, **kwargs)
return cache[key]
return wrapper
return decorator
@memoise_by(lambda n, debug=False: n)
def square(n, debug=False):
return n * n
print(square(4), square(4, debug=True))A custom key function is a statement about which argument differences matter. State that statement in tests, with hits for irrelevant differences and misses for relevant ones, and never let a key silently ignore information the result depends on.
Check: When is a custom cache key function needed, and what happens if it ignores relevant argument information?
Section 42
A context manager for a lock file illustrates how managers can coordinate access to a shared resource across multiple processes. A lock file is a file whose existence signals that a resource is currently in use by another process, and whose absence signals that the resource is free for use by a new process. A lock file manager creates its lock file on entry to its with block and deletes that lock file on exit from its block, whether normally or exceptionally, which guarantees that the lock file is always cleaned up even if an exception occurs inside the block. That guarantee is essential for preventing stale lock files that would block all future processes from accessing the shared resource indefinitely, which is a failure mode that can persist long after the process that caused it has finished.
Lock file managers should handle the case where a lock file already exists when a new process attempts to acquire the lock, by either raising an error indicating that the resource is busy, or by waiting for the lock file to disappear before proceeding with its block. That choice between failing fast and waiting should be documented for users of the manager, so that they know what to expect when a resource is already locked by another process. Failing fast suits batch jobs and scripts that should report contention rather than queue silently, while waiting suits interactive or scheduled work that can afford to queue. Either choice is reasonable, but a manager that sometimes waits and sometimes fails, without documentation, will confuse every caller that encounters contention.
Lock file managers should also handle stale lock files left behind by processes that crashed without cleaning up their lock files, for example by checking the age of an existing lock file and treating very old lock files as stale. That handling needs care, because deleting a lock file that belongs to a live process would break mutual exclusion entirely, allowing two processes to use the resource at once. A safer approach records the owning process identifier in the lock file and checks whether that process is still running before treating the lock as stale, or uses an operating system locking primitive alongside the file where available. Stale-lock handling is therefore a policy decision with real risks, not a simple timeout, and it should be documented alongside the fail-fast or waiting choice.
The example below shows the core structure using exclusive creation, which fails if the file already exists, and guaranteed deletion in a finally clause. It is a teaching sketch of the pattern, and production use should add the ownership and staleness handling described above.
import os
from contextlib import contextmanager
@contextmanager
def lock_file(path):
handle = os.open(path, os.O_CREAT | os.O_EXCL | os.O_WRONLY)
try:
yield
finally:
os.close(handle)
os.unlink(path)
print("lock manager defined")Guarantee deletion on every exit path, document what happens when the lock is already held, and treat stale locks as a safety-critical policy. A lock file manager that leaks its lock is worse than no locking at all, because it blocks everyone while protecting no one.
Check: Why must a lock file manager guarantee deletion on exceptional exit, and how should it handle an already-existing lock file?
Section 43
A decorator that enforces argument types based on a function's annotations illustrates how decorators can use introspection to implement behaviour that depends on a function's own metadata. A function annotated with types, such as def add(a: int, b: int) -> int:, carries those annotations in its __annotations__ attribute, which a decorator can read at definition time to determine what types its arguments should have. That decorator can then check each argument's type on every call to the decorated function, raising a TypeError if any argument has an incorrect type for its corresponding parameter. That checking provides a form of runtime type enforcement for programs that want stricter checking than Python's normal duck typing provides, and it does so without requiring callers to change how they call the function.
Runtime type enforcement through decorators should be used judiciously, because it can break duck typing, as discussed earlier for validation decorators. A function that should accept any object providing required behaviour might instead reject valid objects of unexpected types that nevertheless provide that behaviour, which makes the function less reusable and can break callers that were working correctly. That breakage is a reason to prefer static type checking with tools that analyse code without running it, for most type checking needs in real programs. Static checking catches type errors during development, without imposing runtime costs or runtime restrictions on objects that behave correctly. It also checks code paths that tests may never execute, which runtime enforcement cannot do.
Runtime enforcement should therefore be reserved for cases where invalid types would cause serious harm if not caught early, such as at program boundaries where data enters from external sources, user input, or untrusted callers. At a boundary, a clear TypeError at the point of entry is far better than a confusing failure deep inside the program, or silent corruption of stored data. Inside a program, where data has already crossed a boundary and been validated, repeated runtime checks add cost without adding safety, and they risk rejecting valid objects for no benefit. The decorator's own behaviour should also be documented: which parameters it checks, whether it checks return values as well as arguments, and how it handles subclasses, since a check that rejects subclasses would be surprising for most Python code.
The example below enforces types only for annotated parameters, leaving unannotated parameters unchecked, which mirrors how annotations themselves work: they describe expectations where they are present, and say nothing where they are absent.
from functools import wraps
def enforce_types(func):
annotations = func.__annotations__
@wraps(func)
def wrapper(*args, **kwargs):
bound = list(zip(func.__code__.co_varnames, args))
for name, value in bound:
expected = annotations.get(name)
if expected is not None and not isinstance(value, expected):
raise TypeError(f"{name} must be {expected}")
return func(*args, **kwargs)
return wrapper
@enforce_types
def add(a: int, b: int):
return a + b
print(add(2, 3))Use runtime type enforcement at boundaries where harm is real, prefer static checking inside a program, and document exactly what the decorator checks. Enforcement that rejects valid behaviour is a bug, not a safety feature.
Check: Why is runtime type enforcement through decorators usually reserved for program boundaries, rather than every internal call?
Section 44
Combining multiple state-restoring context managers with ExitStack illustrates how ExitStack can simplify code that needs to change several pieces of global state temporarily for a block of code. Each state change might be managed by a separate context manager, such as one manager for changing the working directory and another manager for setting an environment variable temporarily. Combining those managers in a single with statement with multiple managers would require knowing all managers in advance at code-writing time, while combining them with ExitStack allows managers to be added dynamically based on runtime conditions, such as user configuration or program arguments that determine which state changes are needed for a given execution. That dynamic combination is a flexibility benefit of ExitStack over static with statements with fixed manager lists, and it avoids duplicating the block's body across branches for different combinations of state changes.
That dynamic combination also ensures correct restoration order for multiple state changes, because ExitStack exits managers in reverse order of their entry, so that the last state change applied is the first state change restored when the stack's block exits. That is the correct order for restoring global state that was changed in sequence, because restoring in forward order could restore an incorrect intermediate value for a piece of state that was changed multiple times during entry. Consider a setting changed twice, first to one value and then to another: reverse restoration returns it to the first changed value and then to the original, while forward restoration would leave it at the wrong value. The stack's reverse order therefore mirrors the nesting structure of ordinary with statements, where the innermost block exits first, and it does so even when the set of managers was only decided at runtime.
Using ExitStack for state restoration also makes the cleanup auditable. Every state change is registered in one place, the stack, and every registration is undone when the stack exits, on either exit path. That single point of registration is easier to review than scattered save-and-restore code, where a forgotten restoration in one branch can leak a changed setting into unrelated code. Tests should verify the full combination: that each piece of state has its expected value inside the block, that each piece is restored after a normal exit, and that each piece is restored after an exceptional exit, in the correct order where order matters.
The example below enters two state changes through a stack, in a form that generalises to any number of changes decided at runtime. The stack, not the block's code, owns the restoration.
import os
from contextlib import ExitStack, contextmanager
@contextmanager
def set_env(name, value):
old = os.environ.get(name)
os.environ[name] = value
try:
yield
finally:
if old is None:
del os.environ[name]
else:
os.environ[name] = old
with ExitStack() as stack:
stack.enter_context(set_env("MODE", "test"))
print(os.environ["MODE"])
print("MODE" in os.environ)When the state changes you need are known only at runtime, build them with ExitStack and trust its reverse-order restoration. When they are fixed, a plain with statement is clearer, but the restoration principle is exactly the same.
Check: Why does ExitStack restore multiple state changes in reverse order, and when is dynamic combination better than a fixed with list?
Section 45
An audit trail decorator records information about calls to sensitive functions in a program, such as who called a function, when it was called, with what arguments it was called, and what result it returned or what exception it raised. That audit information is useful for security investigations after an incident, for compliance reporting in regulated industries, and for debugging complex programs where understanding the sequence of sensitive operations performed during a program's execution is essential for diagnosing problems. An audit decorator centralises audit logging logic in one place, so that every sensitive function in a program can be audited consistently by decorating it, without adding audit code inside every sensitive function. Consistency matters for audit trails in particular, because an investigation that finds gaps in the record, where some sensitive calls were logged and others were not, cannot reliably reconstruct what happened.
Audit decorators should be designed carefully, because audit logs themselves can contain sensitive information about a program's operations, such as user identities or data values that should not be visible to unauthorised readers of audit logs. Audit logs should therefore be protected with appropriate access controls and retention policies for their storage, and audit decorators should redact sensitive arguments before logging them, similar to logging decorators discussed earlier in this chapter. Redaction is not optional polish for audit logs; it is part of the security design, because a log that faithfully records every argument, including passwords or personal data, creates a new sensitive store that attackers and unauthorised insiders can target. The decorator should record enough to reconstruct the operation, such as record identifiers and operation types, without recording the sensitive values themselves.
Audit decorators should also log failed calls as well as successful ones, because a security investigation needs to see attempted operations that were denied or that raised exceptions, not only operations that completed. A trail that omits failures hides exactly the events an investigator most wants to find: unauthorised attempts, repeated failures suggesting an attack, and errors that preceded a breach. Failed calls should be logged with the same care as successful ones, including the caller, the time, the arguments in redacted form, and the exception type, while the exception itself should still propagate to the caller unchanged, so that auditing never alters the function's behaviour.
The example below logs both outcomes through the standard logging module, in a try and except structure that records failures before re-raising them. The audit record is a side effect; the function's result and exceptions belong to its caller.
import logging
from functools import wraps
logger = logging.getLogger(__name__)
def audited(func):
@wraps(func)
def wrapper(*args, **kwargs):
try:
result = func(*args, **kwargs)
except Exception as exc:
logger.info("audit: %s failed: %s", func.__name__, type(exc).__name__)
raise
logger.info("audit: %s succeeded", func.__name__)
return result
return wrapper
@audited
def transfer(amount):
return amount
print(transfer(10))Audit every sensitive call, redact sensitive arguments, protect the logs themselves, and never let auditing change what a function returns or raises. A trustworthy audit trail is complete, redacted, and behaviour-preserving.
Check: Why must an audit decorator record failed calls as well as successful ones, and what care is needed with logged arguments?
Section 46
Combining suppress() and redirect_stdout() in a single block illustrates how multiple contextlib managers can be used together to handle complex scenarios where a block of code needs both exception suppression for specific exception types and output redirection for its printed output. A test, for example, may need to capture a piece of code's printed output for assertion purposes, while also suppressing its expected exception for certain inputs. Combining managers in a single with statement with multiple managers separated by commas provides a concise way to express that combination, without needing nested with statements that would be more deeply indented and less clear about the combined behaviour for the block. The single statement reads as one combined setup for the block, which matches how a reader should think about it.
That combination also illustrates the entry and exit order for multiple managers in a single with statement. Managers are entered in order from left to right, and exited in reverse order from right to left, when the block completes or when an exception occurs inside the block. That ordering matters for managers whose setup or cleanup depends on other managers in the same statement, such as a redirection manager that should be set up before a suppression manager that handles exceptions raised during redirected output, or vice versa depending on the specific interaction between those managers for a given use case. In most combinations, the order is not critical, because the managers affect different aspects of execution, but when it is critical, the left-to-right entry and right-to-left exit rule tells you exactly what will happen, and you should choose the order deliberately rather than leaving it to chance.
A further benefit of combining managers in one statement is that cleanup remains guaranteed as a unit. If an exception occurs during entry of a later manager, managers that have already been entered are exited before the exception propagates, so no partially set-up state leaks. If an exception occurs inside the block, all managers exit in reverse order, each performing its own cleanup, and the suppression manager, if it covers that exception type, suppresses it only after the other managers have cleaned up. That choreography is handled by the with statement itself, which is why combining managers in one statement is safer than manually sequencing their setup and cleanup.
The example below combines both managers to capture output while ignoring a missing file. The printed output goes to a buffer, the expected exception is suppressed, and both managers exit cleanly when the block ends.
import io
from contextlib import redirect_stdout, suppress
buffer = io.StringIO()
with redirect_stdout(buffer), suppress(FileNotFoundError):
print("attempting read")
open("missing.txt").read()
print(repr(buffer.getvalue()))When a block needs several kinds of setup at once, list the managers together, order them deliberately where order matters, and let the with statement own their combined cleanup.
Check: How are multiple managers in a single with statement entered and exited, and why does that order matter?
Section 47
Choosing between a decorator and a context manager for a given task is a design decision that should be based on whether the task's behaviour should apply to a function call as a whole, or to a block of code within a function. Decorators are appropriate when behaviour should apply to every call to a specific function, regardless of where that function is called from in a program, such as logging every call to a sensitive function or caching every call to an expensive function. Context managers are appropriate when behaviour should apply to a specific block of code within a function, rather than to every call to that function, such as acquiring a lock for a critical section within a function or redirecting output for a specific block of code within a function. That distinction between function-level and block-level application should guide the choice between the two mechanisms for any given task, and it resolves most cases without further debate.
There are cases where both mechanisms could be used for the same task, such as timing a function, which could be done with a timing decorator applied to that function or with a timing context manager used inside that function's body. The choice between them in such cases should be based on whether timing should apply to every call to that function, which favours a decorator that applies automatically to all calls, or whether timing should apply only to a specific section of the function's body, which favours a manager that scopes the measurement precisely. The same logic applies to logging, retrying, and resource management: scope determines the tool. A decorator's scope is the function, wherever it is called; a manager's scope is the block, wherever it appears.
Scope is not the only consideration, but it is the first and most reliable one. Reuse is the second: a decorator is reused by decorating more functions, while a manager is reused by using it in more blocks, and the form that matches how the behaviour will be reused is usually the clearer one. Explicitness is the third: a decorator's behaviour is visible at the function definition, which helps readers who want to know everything a call does, while a manager's behaviour is visible at the block, which helps readers who want to know what a section does. When scope, reuse, and explicitness all point the same way, the choice is clear; when they conflict, name the conflict explicitly and choose the factor that matters most for that task.
The example below shows both forms side by side for the same timing idea. The decorator times the whole function on every call, while the manager times only the section inside its block, and the difference in scope is visible in the code itself.
import time
from contextlib import contextmanager
from functools import wraps
@contextmanager
def timed_block(name):
start = time.perf_counter()
try:
yield
finally:
print(name, time.perf_counter() - start)
def timed(func):
@wraps(func)
def wrapper(*args, **kwargs):
with timed_block(func.__name__):
return func(*args, **kwargs)
return wrapper
@timed
def work():
return sum(range(100))
print(work())Ask first whether the behaviour belongs to the function or to the block, then let reuse and explicitness confirm the answer. A tool chosen for its scope will rarely need to be revisited.
Check: How does the distinction between function-level and block-level application guide the choice between a decorator and a context manager?
Section 48
Testing with mocking context managers illustrates how context managers can be used to replace external dependencies temporarily during tests. A test might need to replace a function that calls an external service with a mock function that returns predetermined data for testing purposes, without actually calling that external service during testing. The unittest.mock.patch() function provides a context manager for such replacement: it replaces a specified object with a mock object for the duration of its with block, and then restores the original object when its block exits, whether normally or exceptionally. That guarantee is essential for test isolation in programs with many tests that mock different objects for different purposes, because a mock that leaked into a later test would make that test pass or fail for reasons unrelated to its own code.
Mocking context managers should be used sparingly in tests, because overuse of mocking can make tests brittle by coupling them tightly to implementation details of code under test, such as exactly which functions it calls internally, rather than testing its observable behaviour through its public interface. Brittle tests break when implementation details change, even though observable behaviour remains correct for users of the code, which erodes trust in the test suite and discourages refactoring. Mocking should therefore be reserved for external dependencies that cannot be used in tests, such as network services, databases, clocks, or user input, and for dependencies whose real behaviour is slow, unreliable, or difficult to control. Everything that can be tested directly, without a mock, should be tested directly.
When you do mock, mock at the boundary where the dependency is used, not deep inside the implementation, and assert on observable outcomes rather than on the exact sequence of internal calls, wherever possible. A test that patches a service function and checks the result returned to the caller is robust; a test that asserts the precise order of five internal calls is fragile. Also verify restoration, at least occasionally, by checking that the original object is back after the block, especially when tests share a process and a leaked mock could affect unrelated tests. The patch() manager handles restoration for you, but a test that confirms it guards against misuse, such as patching the wrong target and restoring the wrong object.
The example below patches a function for a single block and checks the result. Outside the block, the original function is restored, so later code is unaffected by the test double.
from unittest.mock import patch
def fetch():
return "real data"
def show():
return fetch()
with patch("__main__.fetch", return_value="mock data"):
print(show())
print(show())Reserve mocking for external dependencies, patch at the boundary, assert on observable results, and rely on the manager's restoration guarantee to keep tests isolated. Mocking used sparingly makes tests faster and more reliable; mocking used everywhere makes them brittle.
Check: Why should mocking be reserved for external dependencies, and what guarantee does patch() as a context manager provide?
Section 49
Reviewing decorator and context manager design principles as a whole reveals how they relate to other chapters in this book, such as classes and objects, dunder methods and protocols, iterators and generators, and the standard library. Decorators and managers appear repeatedly as mechanisms for managing cross-cutting behaviour and resources in many different contexts, and that unifying principle explains why Python feels consistent across its built-in types and custom types. Learning decorators and managers, rather than just learning individual examples of them, gives you a deeper understanding of the language as a whole, rather than a list of special cases each to be memorised separately without seeing how they relate to each other and to the language's design. The protocol idea is the thread: a decorator is a callable that transforms a callable, and a manager is an object that follows an entry and exit protocol, and both ideas recur wherever Python needs reusable behaviour.
That perspective will be developed further in later chapters, where decorators and managers are used to build larger, more complex systems whose clarity depends even more on accurate modelling of cross-cutting behaviour between their parts. Those systems include web applications that use decorators for routing and authentication, data processing pipelines that use managers for resource management, and command-line tools that use both mechanisms for logging, timing, and configuration management. In each system, the same questions recur: what behaviour is shared, where does it apply, how is it tested, and how is it documented? Answering those questions well is what separates a system whose cross-cutting behaviour is easy to follow from one where it is hidden in scattered, duplicated code.
The connections to earlier chapters are practical, not just thematic. Classes provide the objects that managers wrap and that class decorators register. Dunder methods define the protocols, __enter__ and __exit__ for managers and the descriptor protocol for built-in decorators, that make the mechanisms work. Generators provide the @contextmanager form, which turns a familiar control structure into a manager. The standard library provides ready-made decorators and managers, from lru_cache to suppress, that embody the same principles in tested form. Seeing those connections means you can transfer what you learned in one chapter to the next, instead of starting from scratch each time.
The example below is deliberately ordinary, because the point of this section is that ordinary language features compose into the patterns this chapter describes. A class, a method, and a call are the building blocks; decorators and managers are ways of organising what happens around them.
class Greeter:
def greet(self, name):
return f"hello {name}"
print(Greeter().greet("ada"))Carry the protocol view forward: look for the entry and exit, the wrapper and the wrapped, and the shared behaviour looking for a single home. That view will make later chapters, and real codebases, substantially easier to read.
Check: How do decorator and manager principles connect this chapter to classes, protocols, and frameworks elsewhere in the book?
Section 50
This final section brings the chapter together by returning to its central idea: decorators and context managers are not isolated tricks, but complementary mechanisms for managing cross-cutting behaviour and resources in Python programs. That perspective transforms how you design programs, from thinking about individual functions or blocks in isolation to thinking about shared behaviour that applies across many functions or blocks. That perspective should be carried forward into every program you write from now on, by asking first whether behaviour should apply at function level or block level, second whether that behaviour is genuinely shared across multiple functions or blocks, third how that behaviour should be tested in its real contexts rather than only in isolation, and fourth how that behaviour should be documented so that users are not surprised by hidden side effects when they use it in their own code.
Those four questions together form a practical design process for decorator and manager based programs that is simple enough to apply routinely and thorough enough to catch most design mistakes before they reach users. Asking about level prevents choosing a decorator for block-scoped work, or a manager for work that should follow a function everywhere. Asking about sharing prevents over-engineering, where a one-off behaviour is wrapped in a reusable component nobody reuses. Asking about testing in real contexts prevents the common failure where a decorator works alone but misbehaves when stacked, retried, or combined with a manager. Asking about documentation prevents the equally common failure where a component works correctly but surprises its users, because its side effects were never stated.
That last point is a reality of decorator and manager design that makes principled thinking essential rather than optional, for any decorator or manager that will be used beyond its author's own code. Users in other programs will assume complete and consistent behaviour based on familiar built-in behaviour, and will be surprised and frustrated when a custom decorator or manager violates those assumptions silently, without documentation or warning. Principle-based design scales to new contexts and new requirements, while ad hoc decisions do not scale and must be revisited for every new context, with additional cost and additional risk each time a new context reveals a gap or inconsistency that principled design would have caught from the start, by asking the right questions about behaviour before implementation began and by documenting the answers for future maintainers.
The short example below closes the chapter with the same reminder the checkpoint sections gave: clear names and single responsibilities are the foundation that every larger pattern builds on, and they cost nothing to adopt.
class Book:
def __init__(self, title):
self.title = title
print(Book("Dune").title)Keep the four questions beside you for every new decorator or manager you design: level, sharing, testing in context, and documentation. Principles scale better than ad hoc decisions, because they answer the next question before it is asked, and they let maintainers extend your work without rediscovering its promises.
Check: What four questions should you ask when designing decorator or manager behaviour, and why do principles scale better than ad hoc decisions?
Practical Task
Build a small timing-and-retry toolkit in one file. Write a @timed decorator, using functools.wraps, that prints a function's name and elapsed time measured with a monotonic clock, even when the function raises, and a @retry(times) decorator factory that retries only ValueError, with a short delay, and re-raises after the final attempt. Write a @contextmanager function called temporary_directory that changes into a given directory on entry and restores the original directory on exit, even on exception. Write tests, using plain assert statements and a recording stub function, that check: the timed decorator preserves the function's name; the retry decorator succeeds after two failures and raises after exhausting its attempts; the retry decorator does not retry a TypeError; and the directory manager restores the original directory after both normal exit and an exception inside the block. Also verify that stacking @timed outside @retry logs every attempt, while stacking it inside logs only the final outcome once.
Chapter Questions
- Why does a decorator with arguments require a factory that returns a decorator, and when does each of the three nested functions run?
- How does
functools.wrapspreserve metadata, and why does its absence compound through stacked decorators? - When should you implement a context manager as a class with
__enter__and__exit__, and when is a@contextmanagergenerator function clearer? - Why should a retry decorator retry only specific exception types, and what danger applies to retrying non-idempotent operations?
- How does the distinction between function-level and block-level application guide the choice between a decorator and a context manager for a given task?