Writing

Filtering and Sorting Data with Polars: Expressions, Strings, and Multi-Column Ordering

In data analysis, isolating precise records and ordering outputs are core building blocks. In Polars, both operations are powered by the Expression API, enabling the query planner to evaluate filters in parallel and execute predicate pushdown directly against underlying storage.

Download the example data: Pokemon.csv.

1. Boolean Filtering with filter()

Unlike Pandas where boolean conditions often require complex slicing syntax (df[(df['A'] > 1) & (df['B'] < 2)]), Polars evaluates boolean expressions cleanly inside .filter():

import polars as pl

df = pl.read_csv("Pokemon.csv")

# Single condition
high_hp = df.filter(pl.col("HP") >= 100)

Combining Multiple Conditions

Combine expressions with bitwise operators: & (AND), | (OR), and ~ (NOT). Remember to enclose sub-expressions in parentheses to enforce correct operator precedence:

# Multiple AND conditions
strong_fire = df.filter(
    (pl.col("Type 1") == "Fire") & (pl.col("Attack") >= 100)
)

# OR condition
special_targets = df.filter(
    pl.col("Legendary") | (pl.col("HP") >= 150)
)

# Negation and membership checks (is_in)
standard_types = df.filter(
    ~pl.col("Type 1").is_in(["Fire", "Water"])
)

Handling Missing Values (is_null, is_not_null)

Polars represents missing data with strict null semantics (rather than floating-point NaNs mixed with objects):

# Filter for missing values (single-type Pokemon without secondary type)
single_type = df.filter(pl.col("Type 2").is_null())

# Filter for existing values (dual-type Pokemon)
dual_type = df.filter(pl.col("Type 2").is_not_null())

2. String Pattern Filtering

The .str namespace provides vectorized string predicates:

# Contains substring
mega_pokemon = df.filter(pl.col("Name").str.contains("Mega"))

# Starts with prefix
starts_with_p = df.filter(pl.col("Name").str.starts_with("P"))

# Regex matching (case-insensitive flag)
ends_with_saur = df.filter(
    pl.col("Name").str.contains(r"(?i)saur$")
)

3. Sorting Data with sort()

Polars provides high-performance multi-threaded sorting algorithms.

Single and Multi-Column Sorting

# Descending sort by single column
hp_sorted = df.sort("HP", descending=True)

# Multi-column sort with distinct sort directions:
# Sort by Type 1 ascending, then by Attack descending within each type
type_attack_sorted = df.sort(
    ["Type 1", "Attack"],
    descending=[False, True]
)

Sorting by Dynamic Expressions

You do not need to materialize temporary columns to sort by calculated values. Pass expressions directly into sort():

# Sort by combined Attack + Defense power
power_sorted = df.sort(
    pl.col("Attack") + pl.col("Defense"),
    descending=True
)

End-to-End Pipeline Example

Chaining filtering, computed columns, sorting, and projection into an idiomatic Polars pipeline:

strongest_non_legendary = (
    df
    .filter(~pl.col("Legendary"))
    .with_columns(
        (
            pl.col("HP")
            + pl.col("Attack")
            + pl.col("Defense")
            + pl.col("Sp. Atk")
            + pl.col("Sp. Def")
            + pl.col("Speed")
        ).alias("Total Stats")
    )
    .sort("Total Stats", descending=True)
    .select(["Name", "Type 1", "Type 2", "Total Stats"])
    .head(5)
)

print(strongest_non_legendary)
shape: (5, 4)
┌──────────────┬────────┬────────┬─────────────┐
│ Name         ┆ Type 1 ┆ Type 2 ┆ Total Stats │
│ ---          ┆ ---    ┆ ---    ┆ ---         │
│ str          ┆ str    ┆ str    ┆ i64         │
╞══════════════╪════════╪════════╪═════════════╡
│ Slaking      ┆ Normal ┆ null   ┆ 670         │
│ Tyranitar    ┆ Rock   ┆ Dark   ┆ 600         │
│ Salamence    ┆ Dragon ┆ Flying ┆ 600         │
│ Metagross    ┆ Steel  ┆ Psychic┆ 600         │
│ Garchomp     ┆ Dragon ┆ Ground ┆ 600         │
└──────────────┴────────┴────────┴─────────────┘

Next in the series: Joining and Reshaping Data with Polars.