System resources—such as open file descriptors, network sockets, database sessions, and locks—are finite. When an application allocates a resource but fails to release it, it causes resource leaks, eventually degrading performance or exhausting operating system handles.
The Problem with Manual Cleanup
Consider opening a file and reading its content:
f = open('memo.txt', 'r')
print(f.read())
f.close()
While clean on the surface, this pattern is brittle. If an unexpected exception occurs between open() and f.close(), the file handle remains unclosed until the garbage collector eventually runs:
f = open('memo.txt', 'r')
data = f.read()
process_data(data) # Suppose an unhandled exception occurs here
f.close() # Never executed!
Using try...finally guarantees cleanup:
f = open('memo.txt', 'r')
try:
data = f.read()
process_data(data)
finally:
f.close() # Guaranteed to execute regardless of errors
While effective, repeating try...finally everywhere introduces visual noise and error-prone boilerplate.
The Idiomatic Solution: The with Statement
Python's with statement encapsulates acquisition and guaranteed release into a concise, readable block:
with open('memo.txt', 'r') as f:
data = f.read()
process_data(data)
# File is automatically closed upon leaving this block
Even if process_data() raises an unhandled exception, Python guarantees the file is safely closed before the exception bubbles up.
Building Custom Context Managers (Class Protocol)
Any object can participate in a with statement by implementing two dunder methods:
__enter__(self): Invoked upon entering thewithblock. Its return value is bound to the identifier specified afteras.__exit__(self, exc_type, exc_val, exc_tb): Invoked when leaving the block. If an exception occurred, its type, value, and traceback are passed as arguments. ReturningTruesuppresses the exception; returningNoneorFalseallows it to propagate.
Here is an example managing a database session with SQLAlchemy:
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
class DatabaseSession:
"""Manages transactional database session lifecycle."""
def __init__(self, connection_string: str):
self.engine = create_engine(connection_string)
self.SessionFactory = sessionmaker(bind=self.engine)
self.session = None
def __enter__(self):
self.session = self.SessionFactory()
return self.session
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is not None:
self.session.rollback()
self.session.close()
Usage:
db = DatabaseSession("sqlite:///app.db")
with db as session:
user = session.query(User).filter_by(id=1).one_or_none()
print(user.name)
The Lightweight Alternative: @contextmanager
When building a full class feels unnecessarily heavy, Python's standard library provides contextlib.contextmanager. This decorator converts a generator containing a single yield statement into a context manager:
from contextlib import contextmanager
@contextmanager
def managed_resource(*args, **kwargs):
# Setup / Acquisition
resource = acquire_resource(*args, **kwargs)
try:
yield resource
finally:
# Teardown / Release
release_resource(resource)
Everything before yield runs on __enter__, the yielded value is bound to as, and everything in finally runs on __exit__.
Asynchronous Context Managers (async with)
Modern asynchronous code using asyncio often requires managing connections or locks asynchronously. Python provides:
__aenter__and__aexit__for class-based async contexts.contextlib.asynccontextmanagerfor generator-based async contexts.
from contextlib import asynccontextmanager
@asynccontextmanager
async def get_db_connection():
conn = await acquire_db_connection()
try:
yield conn
finally:
await release_db_connection(conn)
async def get_all_users():
async with get_db_connection() as conn:
return await conn.query("SELECT * FROM users")
Best Practices
- Keep context blocks narrow: Only enclose lines that strictly depend on the scoped resource.
- Never swallow unexpected exceptions: In
__exit__, only returnTrueif you deliberately intended to catch and handle the specific error. - Favor
@contextmanagerfor simple state swaps: Temporarily modifying environment variables or redirecting stdout can be done in 5 lines withcontextlib.