Domain Explainer: Caching Libraries#

For readers deciding whether this category is relevant to them, and what they need to know before reading the passes.

The whole answer, first#

A cache keeps the answer to a question so the next time it is asked, nobody has to work it out again. Finding the answer already there is a hit; not finding it — and having to work it out and store it — is a miss. In Python there are three places to keep a cache, and the place decides the library. Measured for the survey on one machine, a hit on the cheapest possible function:

The fourth number is the one the category hides: what hit rate will your traffic give the cache? The hit rate is the share of requests the cache answers, and it is a property of the traffic, not the library. A cache is always smaller than the set of things it could hold, so when it is full it must throw something out, and the rule for choosing is its eviction policy: LRU (least recently used) discards whatever was asked for longest ago; LFU (least frequently used) whatever was asked for fewest times; FIFO (first in, first out) whatever arrived earliest; random, at random. On a stable hot set — a few keys getting most requests — with a cache holding 10% of the keys, LRU gets 67%. When the favourites move, LRU keeps 66% and LFU drops to 49%. On a loop over more keys than fit, LRU gets 0% — at any size up to half.

That is the finding. The rest of this page is why, and all of it runs in your browser at /workshop/cache-me/.

The one-paragraph version#

Caching looks like one decision and is three. The first is the tier: who reads the cache — one process, one machine, or many — and that is set by your architecture before any library is chosen. The second is the miss: every tier makes hits cheap, and charges for misses — the function and the store, which on a disk is a write and on a network is a second trip — so a cache on a fast function with a middling hit rate makes the code slower, and the break-even is a number you can compute. The third is the policy, and it matters less than people think on ordinary traffic and more than they think on two kinds: traffic whose favourites move (LRU recovers, LFU does not) and traffic that loops over more than fits (nothing recovers). The libraries are all good; the three decisions are made before any of them is chosen.

What a cache does, and what it costs#

The in-process tier. functools.lru_cache is a C dictionary keyed on the function’s arguments, evicting the least recently used entry past a count. A hit is a hash lookup and a pointer move: 0.03 µs. It cannot expire an entry on a clock and cannot bound itself by bytes. cachetools can — TTLCache, a getsizeof weight, LRUCache / LFUCache / FIFOCache / RRCache, caches you can inspect and clear — and it is pure Python, so a hit is a decorator, a key, a method call and, for TTL, a clock read: 0.55–1.2 µs. Twenty to forty times the standard library, and invisible next to any function worth caching; a local database query is a thousand of those microseconds and an HTTP call ten thousand. The one case the overhead decides is a function that costs less than about 10 µs, and the answer there is not to cache it.

The on-machine tier. diskcache keeps the cache in SQLite — a database that lives in an ordinary file — so it survives a restart, is shared by every process on the box, and needs no daemon. A read is 4 µs. A write is a SQLite transaction: 350 µs on a real disk, and 18 µs per key when two hundred of them share one transact(). That asymmetry sets its break-even: a 100 µs function needs an 82% hit rate for diskcache to pay; a 10 ms one needs 4%. It is the tier for results that are expensive — queries, downloads, model artifacts — on one machine.

The network tier. redis-py talks to a Valkey or Redis server; pymemcache to memcached. Every hit and every miss is a round trip. Measured with the network taken out — over a unix socket, a connection between two processes on the same machine that skips the network stack — one synchronous Python call is 106 µs median with redis-py and 71 µs with pymemcache; the raw floor under both is 11 µs, so the clients cost 35 and 3 µs respectively and the medians are the operating system’s. Over this machine’s TCP loopback the same call is 320 µs; a real network adds its own latency on top. A pipeline — one trip for a hundred keys — makes every client 3–10 µs per key, and is the single order-of-magnitude lever on this tier. The servers are not the difference: Valkey 9.1.1 and Redis 8.10.1 measure the same behind the same client.

The policy question#

The survey replayed 30,000 requests over 10,000 keys through cachetools’ own cache classes, one per policy, with a cache holding 10% of the keys, and compared them with the ceiling — the best any policy could do if it could see the future:

On a stable hot set the choice barely matters, and a cache holding one key in a hundred already catches two requests in five — the first few percent of cache do most of the work. When the hot set moves, LRU forgets in one cache-length and LFU keeps counting the old favourites: 16 points. A scan through everything pollutes LRU (it dutifully caches each scanned key once) and not LFU; newer policies that admit a key only once it has proved itself — W-TinyLFU, ARC — exist for that case, and cachetools does not ship one. With no favourites the hit rate is the cache size and no policy changes it. And a loop larger than the cache defeats LRU and FIFO completely: each key is evicted one step before it comes round again, every pass, at every size up to half. The only remedies are a cache at least as large as the loop, or no cache.

One more thing, two lines long: the textbook sequence on which a bigger FIFO cache misses more — Belady’s anomaly — reproduces in cachetools’ FIFOCache (9 misses with 3 slots, 10 with 4). LRU cannot do that. It is the reason the default is LRU and not FIFO.

The server question, since 2024#

Behind redis-py there are now two servers. Redis 7.4 left the BSD license for RSALv2 / SSPLv1 in March 2024; the Linux Foundation forked Redis 7.2.4 as Valkey (BSD-3-Clause) the same month; AWS priced ElastiCache for Valkey 20–33% below Redis OSS that October; Redis 8 added AGPLv3 as a third option in May 2025. The wire protocol is the same, the client is the same, the measured round trip is the same. What differs is the license your lawyer will accept, the managed SKU your cloud offers, and one door that only opens one way: Valkey reads data files written by Redis 7.2 and earlier, not 7.4 or later. Decide before that upgrade, not after.

What to do#

1. Name the tier. Who reads the cache — one process, one machine, many? That is the library.
2. Cost the miss. What does the function cost, what does the store cost on that tier, and what hit rate does that imply for break-even? The floor model computes it for the cost you set.
3. Measure the pattern. Replay a day of keys through the policies before choosing one — or accept LRU, which is the default for a good reason.
4. Never fetch two keys in two trips on the network tier.
5. Batch writes on diskcache.

And if the traffic is a loop over more than fits, the right amount of cache is either all of it or none.

S1 - Rapid Discovery: Caching Libraries#

Every figure measured 2026-08-20 for the cache-me floor model (/workshop/cache-me/). Download counts are pypistats.org, last 30 days, read 2026-08-20. Timings are what a Python caller sees on one machine (CPython 3.14.7, aarch64, single thread); the method and the full tables are in S2 §0, and the bench imports the page’s own core.py.

The pick#

Three tiers, one library each — and a question to answer before any of them.

from functools import lru_cache
@lru_cache(maxsize=4096)                       # 0.03 µs a hit; start here
def price(sku): ...

import cachetools
@cachetools.cached(cachetools.TTLCache(4096, ttl=300))   # 1.2 µs a hit, and it expires
def rate(currency): ...

import diskcache
cache = diskcache.Cache("/var/cache/app")
@cache.memoize(expire=3600)                    # 12 µs a hit; survives a restart
def report(day): ...
with cache.transact():                         # 18 µs a write instead of 350
    for k, v in bulk: cache.set(k, v)

import redis
r = redis.Redis(host="cache")                  # Valkey or Redis — same client, same number
r.setex(f"user:{uid}", 3600, blob)             # 106 µs a trip
with r.pipeline(transaction=False) as p:       # 7 µs a key
    for k in keys: p.get(k)
    values = p.execute()

The question before the library: what hit rate will your traffic give it?#

30,000 requests over 10,000 keys, a cache holding 10% of them, replayed through cachetools’ own classes. The numbers are deterministic and the floor model re-runs them on any pattern and size.

Three things to read off it. On a stable hot set the policies are within a few points and a cache holding 1% of the keys already catches 39% of requests. When the favourites move, LRU follows and LFU keeps counting yesterday’s — 16 points. And on a loop that does not fit, LRU and FIFO score exactly zero at every size up to half: each key is evicted one step before it comes round again. No better LRU fixes that; only a cache at least as big as the loop, or none.

What a hit costs, what a miss costs#

A cache pays when the hit rate clears store ÷ (function + store − hit). With diskcache’s 350 µs write, a 100 µs function needs 82% hits to break even and a 10 ms one needs 4%; over a socket, a 100 µs function needs 52%. The floor model’s third panel does this arithmetic for the function cost you set.

Downloads a month: cachetools 339M, redis 275M, diskcache 37M, valkey 5.1M, pymemcache 3.3M, dogpile.cache 2.5M, python-memcached 1.3M, valkey-glide 1.0M.

Three things that decide most choices#

The tier decides before the library. Inside one process the standard library is the fastest cache in Python and the one to start with; cachetools is the same tier with a clock and a weight. diskcache is a different tier — persistence without a server — and the network is a third. Nothing in a lower tier can be made to serve a higher one, and nothing in a higher one is as cheap.

The miss is the price. Hits are cheap everywhere. What a tier charges is the store after a miss and, on the network, the trip either way. Put a number on the function before adding the cache; if it costs under ten microseconds, the decorator is the cost.

Redis and Valkey are the same number. Behind redis-py, 106 µs and 110 µs. The choice between them is license (Redis 8: RSALv2 / SSPLv1 / AGPLv3; Valkey: BSD-3-Clause) and the cloud bill (ElastiCache for Valkey is 20–33% cheaper), and it is one-way past Redis 7.2 because the data-file format diverged. S2 §0 has the dates, the versions and the three clients.

When each one stops being a reason#

Install#

# functools.lru_cache — standard library
pip install cachetools          # pure Python, MIT, 7.1.7
pip install diskcache           # pure Python over SQLite, Apache-2.0, 5.6.3
pip install redis               # redis-py, MIT, 8.1.0 — speaks to Valkey and Redis
pip install pymemcache          # Apache-2.0, 4.0.0
pip install dogpile.cache       # MIT, 1.5.0

Measured: 2026-08-20

S2 Comprehensive Discovery: Caching Libraries#

Measured: 2026-08-20 (§0); library notes and patterns below Methodology: S2 - Systematic technical evaluation across performance, features, and ecosystem

0. Measured 2026-08-20 — what building the floor model established#

Everything in this section was measured, not read: one machine — CPython 3.14.7, aarch64 (Qualcomm Snapdragon X under WSL2), single thread — with the bench importing the floor model’s own core.py (01-discovery/bench/, results in results.json), and the in-process and on-disk tiers run again in the browser at /workshop/cache-me/ (Pyodide 314.0.5, Python 3.14.2) at about a third of native speed, same shape, identical hit rates. Versions: cachetools 7.1.7, diskcache 5.6.3, redis-py 8.1.0, valkey-py 6.1.1, valkey-glide 2.5.1, pymemcache 4.0.0, dogpile.cache 1.5.0, cachebox 6.2.5; servers Valkey 9.1.1, Redis 8.10.1 and memcached 1.6.38 built from source on the same machine. The server and client license facts were read from the projects' own pages the same day (§0.5).

0.1 What a hit costs, what a store costs#

Each cache wraps the cheapest possible function, k + 1, so the figure is the cache’s own overhead. 1,000 warmed keys, 1,000 calls per batch, best-of under a budget.

In the browser (Pyodide, MEMFS): lru_cache 0.15 µs, cachetools LRU 1.85 µs, TTL 3.4 µs, diskcache get 6.6 µs — same order, roughly 3× slower. diskcache set is 55 µs there because nothing is fsynced; the page says so and shows the disk figure in orange.

Memory per entry (tracemalloc, 10,000 int → int): dict 61 B, lru_cache 116 B, LRUCache 135 B, LFUCache 142 B, TTLCache 223 B. The TTL link costs a dict’s worth again.

0.2 Hit rate by policy and pattern#

30,000 requests over 10,000 keys, seed 7, replayed through cachetools’ own classes with __getitem__ (so a hit updates recency), against functools.lru_cache (identical to LRUCache on every row) and Belady’s offline MIN as the ceiling. “Hot set” is Zipf, s = 1, over a shuffled key space; “moves” re-shuffles the favourites every third of the run; “plus a scan” mixes one sequential request in four.

Belady’s anomaly, the sequence 1 2 3 4 1 2 5 1 2 3 4 5 through cachetools: FIFOCache 9 misses with 3 slots, 10 with 4; LRUCache 10 → 8.

0.3 The network tier, per operation#

Per-operation min / median / p90 over 3,000 calls, one synchronous Python client. “unix” is the client’s own cost with the network removed; “tcp” adds this machine’s loopback stack, which under WSL2 is a 112 µs floor by itself. The raw rows are a PING on a bare socket — what the machine charges before any client code runs.

What the table says: the server is not the difference (Valkey and Redis within 3 µs behind the same client); the client is a difference of tens of microseconds at the minimum (pymemcache 3 µs over the floor, valkey-py 24, redis-py 34 — redis-py without the hiredis parser) and none at the median; the operating system is most of the median; and a pipeline is the one lever that changes the tier’s cost by an order of magnitude. valkey-glide’s single-call figure includes an event-loop turn per await — its case is concurrency, and with 100 awaits in flight it is 27 µs per key. pymemcache’s set() is noreply=True by default and returns in 2 µs without a round trip; the SET above asked for the reply.

0.4 Break-even, from the measured costs#

core.payoff(): mean time with cache = h·hit + (1 − h)·(fn + store); the cache pays when h > store ÷ (fn + store − hit).

0.5 The servers and clients behind the network tier (read, not measured)#

Read on 2026-08-20 from GitHub release lists via the API, redis.io/legal/licenses, valkey.io/topics/migration, the Valkey 9.0 and 9.1 release posts, the AWS “what’s new” notice for ElastiCache for Valkey and the Memorystore for Valkey product page.

Valkey 9.0 (2025-10-21): numbered databases in cluster mode, atomic slot migration, per-field hash expiry (HEXPIRE, HTTL, HPERSIST), DELIFEQ, polygon geosearch, and the project’s headline — over one billion requests per second across 2,000 nodes. 9.1 (2026-05-19): 2.1 million req/s on a single server (512-byte payloads, 9 I/O threads, pipeline depth 10), up to 20% less memory for strings under 128 bytes, ACLs scoped to a database. Redis 8 answered with vector sets and the bundled Stack modules. On the cache path — GET, SET, SETEX, EXPIRE, pipelines, pub/sub — they are the same server with different licenses, and §0.3 measures them as such.

Comprehensive Library Analysis#

1. redis-py (Redis Python Client)#

Technical Specifications:

• Performance: one round trip per call — 106 µs median over a unix socket, 320 µs over TCP loopback on the bench machine (§0.3); 7 µs per key pipelined
• Memory: server-side; RESP3 on the wire by default since 8.0
• Features: Pub/sub, transactions, clustering, persistence
• Ecosystem: Extensive tooling, monitoring, cloud services

Strengths:

• Industry-proven scalability (Instagram, GitHub, Twitter)
• Rich data structures (strings, hashes, lists, sets, sorted sets)
• Built-in persistence and high availability
• Extensive monitoring and operational tools
• Active development and enterprise support

Weaknesses:

• Higher memory overhead than pure cache solutions
• Network latency for distributed setups
• Complexity for simple use cases
• Additional infrastructure dependency

Best Use Cases:

• Multi-server applications requiring shared state
• Real-time features (leaderboards, counters, sessions)
• Complex data structures beyond key-value pairs
• Applications requiring persistence and high availability

2. python-memcached / pymemcache#

Technical Specifications:

• Performance: one round trip per call — 71 µs median over a unix socket, 14 µs at best (§0.3); 2.9 µs per key with get_many
• Memory: server-side, slab-allocated; the client is pure Python
• Features: Simple key-value storage, LRU eviction
• Ecosystem: Mature, lightweight, focused

Strengths:

• Fastest pure caching performance
• Minimal memory overhead
• Battle-tested stability (Facebook, Wikipedia)
• Simple operational model
• Predictable behavior under load

Weaknesses:

• No persistence (data lost on restart)
• Limited data structures (key-value only)
• No built-in clustering or replication
• Limited observability features

Best Use Cases:

• High-frequency API response caching
• Session storage for stateless applications
• Database query result caching
• Maximum performance requirements

3. diskcache#

Technical Specifications:

• Performance: get 4 µs, memoize 12 µs per hit; set 350 µs on disk, 18 µs per key inside one transact() (§0.1)
• Memory: minimal in-process; SQLite-backed persistence
• Features: TTL, LRU, size limits, thread-safe operations
• Ecosystem: Zero dependencies, pure Python

Strengths:

• Persistent across application restarts
• No external infrastructure required
• Thread-safe and process-safe operations
• Built-in eviction policies
• Excellent for development and single-server deployments

Weaknesses:

• Slower than memory-based solutions
• Not suitable for distributed applications
• Filesystem I/O limitations
• Limited concurrent access performance

Best Use Cases:

• Single-server applications
• Development environments
• Caching large objects or files
• Applications requiring cache persistence

4. cachetools#

Technical Specifications:

• Performance: LRU 0.55 µs, LFU 0.8 µs, TTL 1.2 µs per hit (§0.1) — against 0.03 µs for functools.lru_cache
• Memory: 135 B per entry (LRU), 223 B (TTL), against 61 B for a dict and 116 B for lru_cache
• Features: LRU, TTL, decorators, multiple eviction strategies
• Ecosystem: Stdlib-style API, decorator patterns

Strengths:

• Zero external dependencies
• Decorator-based usage patterns
• Multiple cache strategies (LRU, TTL, LFU)
• Perfect for function memoization
• Immediate implementation

Weaknesses:

• Single-process only
• Memory limited by Python process
• No persistence across restarts
• Limited observability

Best Use Cases:

• Function result caching
• Single-process applications
• Prototype development
• Simple in-memory caching needs

5. dogpile.cache#

Technical Specifications:

• Performance: Backend-dependent, abstraction overhead
• Memory: Backend-dependent
• Features: Multi-backend, regions, key generation, decorators
• Ecosystem: SQLAlchemy integration, enterprise features

Strengths:

• Backend abstraction (Redis, Memcached, files, database)
• Advanced features (regions, key namespacing, decorators)
• SQLAlchemy integration for ORM caching
• Enterprise-grade locking and dogpile prevention
• Flexible configuration management

Weaknesses:

• Additional abstraction layer overhead
• Complexity for simple use cases
• Learning curve for advanced features
• Smaller community compared to direct backend libraries

Best Use Cases:

• Complex applications with multiple caching needs
• SQLAlchemy/ORM-heavy applications
• Enterprise applications requiring sophisticated caching strategies
• Applications needing backend flexibility

Performance Comparison Matrix#

Speed (measured, §0.1 and §0.3):#

Memory Efficiency:#

Feature Comparison:#

Ecosystem Analysis#

Community and Maintenance:#

• redis-py: Very active, Redis Inc. backing (MIT-licensed; connects to Redis and Valkey alike), extensive documentation
• pymemcache: Pinterest-maintained, stable, focused scope
• diskcache: Grant Jenks maintained, regular updates, good documentation
• cachetools: Thomas Kemmer maintained, stable, minimal changes needed
• dogpile.cache: Mike Bayer (SQLAlchemy) maintained, enterprise focus

Production Readiness:#

• redis-py: Enterprise-proven, extensive operational tooling
• pymemcache: Battle-tested at Pinterest, Wikipedia scale
• diskcache: Reliable for single-server use cases
• cachetools: Simple and stable, good for contained use cases
• dogpile.cache: Enterprise-ready, complex deployment scenarios

Integration Patterns:#

• redis-py: Often combined with Redis Cluster, Redis Sentinel
• pymemcache: Typically used with load balancers, consistent hashing
• diskcache: Standalone or with application-level coordination
• cachetools: Function-level integration, decorator patterns
• dogpile.cache: Framework integration, especially with SQLAlchemy

Architecture Patterns and Anti-Patterns#

Recommended Patterns:#

Multi-Tier Caching:#

# L1: In-memory for hot data
@cachetools.cached(cachetools.TTLCache(maxsize=100, ttl=60))
def hot_data(key):
    # L2: Redis for shared data
    result = redis_client.get(f"shared:{key}")
    if result:
        return json.loads(result)

    # L3: Database for persistent data
    result = database.query(key)
    redis_client.setex(f"shared:{key}", 300, json.dumps(result))
    return result

Cache-Aside Pattern:#

def get_user_profile(user_id):
    # Check cache first
    cached = redis_client.get(f"user:{user_id}")
    if cached:
        return json.loads(cached)

    # Load from database
    profile = database.get_user(user_id)

    # Update cache
    redis_client.setex(f"user:{user_id}", 3600, json.dumps(profile))
    return profile

Write-Through Caching:#

def update_user_profile(user_id, data):
    # Update database
    database.update_user(user_id, data)

    # Update cache immediately
    redis_client.setex(f"user:{user_id}", 3600, json.dumps(data))

Anti-Patterns to Avoid:#

Cache Stampede (Multiple requests regenerating same data):#

# BAD: No protection against simultaneous cache misses
def expensive_operation(key):
    result = cache.get(key)
    if not result:
        result = very_expensive_computation()  # Multiple threads might run this
        cache.set(key, result, ttl=300)
    return result

# GOOD: Use locking or single-flight pattern
import threading
_locks = {}

def expensive_operation(key):
    result = cache.get(key)
    if not result:
        lock = _locks.setdefault(key, threading.Lock())
        with lock:
            result = cache.get(key)  # Double-check
            if not result:
                result = very_expensive_computation()
                cache.set(key, result, ttl=300)
    return result

Cache Invalidation Race Conditions:#

# BAD: Data modification without proper cache invalidation
def update_data(key, new_data):
    database.update(key, new_data)
    # Race condition: cache might be repopulated with old data here
    cache.delete(key)

# GOOD: Atomic operations or versioning
def update_data(key, new_data):
    with database.transaction():
        database.update(key, new_data)
        cache.delete(key)

Selection Decision Framework#

Use redis-py when:#

• Multi-server application architecture
• Need pub/sub, transactions, or complex data structures
• Require persistence and high availability
• Team has Redis operational expertise
• Budget allows for a server to run, or a managed SKU

Use pymemcache when:#

• Maximum caching performance required
• Simple key-value caching sufficient
• Distributed caching needed but Redis features unnecessary
• Cost optimization important (cheaper than Redis)
• Existing Memcached infrastructure

Use diskcache when:#

• Single-server deployment
• Need cache persistence across restarts
• Zero additional infrastructure desired
• Development or staging environments
• File-based caching acceptable performance

Use cachetools when:#

• Single-process application
• Function result memoization primary use case
• Minimal complexity preferred
• Prototype or development phase
• No external dependencies allowed

Use dogpile.cache when:#

• Complex multi-backend caching requirements
• Heavy SQLAlchemy/ORM usage
• Enterprise features needed (regions, advanced invalidation)
• Backend flexibility important for future changes
• Team has expertise in advanced caching patterns

Technology Evolution and Future Considerations#

Current direction (2026):#

• Two servers, one client: Valkey (BSD) and Redis (RSALv2 / SSPLv1 / AGPLv3) behind the same redis-py; managed SKUs on AWS and GCP default to Valkey (§0.5)
• RESP3 by default in redis-py 8 — the transport for server-assisted client-side caching
• Compiled in-process caches (cachebox, Rust) at 0.28 µs per hit, between the stdlib and cachetools
• Scan-resistant policies (W-TinyLFU in theine, ARC) for the mixed-traffic case where LRU loses to LFU
• Full viability analysis in S4

Conclusion#

The caching library ecosystem offers clear specialization:

1. Redis dominates distributed caching with rich features and proven scalability
2. Memcached leads pure performance for simple key-value caching
3. DiskCache excels for single-server persistent caching needs
4. cachetools provides simplicity for in-process function memoization
5. dogpile.cache handles complexity for enterprise multi-backend scenarios

Recommended approach: Start with cachetools for immediate gains, evolve to Redis for distributed needs, consider specialized solutions (Memcached, DiskCache) for specific performance or deployment constraints.

S3 Need-Driven Discovery: Caching Libraries#

Measured: 2026-08-20, for the cache-me floor model (/workshop/cache-me/). Numbers are from S2 §0 (CPython 3.14.7, one machine); the hit rates are deterministic and re-run in any browser at the floor model.

The three questions every persona answers#

1. Which tier? One process (functools.lru_cache, cachetools), one machine across restarts (diskcache), or many machines (redis-py against Valkey or Redis, pymemcache against memcached). The tier is set by where the readers of the cache run, and it decides before the library does.
2. What does a miss cost? A hit is cheap on every tier — 0.03 µs in-process, 4 µs on disk, 50–100 µs across a socket. The miss pays for the function and the store, and on diskcache the store is 350 µs on a real disk. The cache pays when the hit rate clears store ÷ (function + store − hit); the floor model’s third panel computes it.
3. What is the access pattern? The hit rate belongs to the traffic, not the library. On a stable hot set every policy is within a few points; when the hot set moves LRU beats LFU by 16; on a loop bigger than the cache LRU and FIFO score exactly zero.

Use Case 1: Several application servers, one answer#

Who: A team running a web API on more than one process or host, where the same expensive result — a rendered fragment, a computed listing, a third-party lookup — is wanted by every instance.

Why it is a caching decision: In-process caches fill once per instance and cannot be invalidated together. The requirement is a cache the instances share, so a miss on one becomes a hit on all.

# Requirements
# - shared across processes and hosts
# - a time-to-live, so a stale entry expires without anyone remembering to delete it
# - a round trip cheap next to the thing being cached
# - one server to run, or one managed service to pay for

Answer: redis-py with SETEX, against whichever of Valkey and Redis the license and the cloud bill favor (S2 §0 — they are the same number behind the same client). Pipeline anything that fetches more than one key. memcached is the right pick when the team wants nothing else from the server and wants to say so.

Use Case 2: One machine, expensive results that must survive a restart#

Who: A data engineer or analyst whose job recomputes costly intermediate results — query results, feature tables, downloaded artifacts — on one box, and restarts the process often.

Why it is a caching decision: An in-process cache dies with the process; a server is a second thing to run for a one-machine job. The requirement is persistence without infrastructure.

# Requirements
# - survives a restart
# - handles large values (100 KB – 10 MB)
# - no server
# - a miss is a query that takes milliseconds to minutes, so store cost is tolerable

Answer: diskcache. Its memoize() decorator is 12 µs per hit, and with a 10 ms function the cache breaks even at a 4% hit rate. Wrap bulk loads in with cache.transact(): — the single-write cost is SQLite’s transaction, and one transaction around 200 writes divides it by twenty.

Use Case 3: Sessions, counters and rate limits across instances#

Who: The same multi-instance web team as Use Case 1, but the data is small, written often, and correctness matters: a session token, a login-attempt counter, a per-user rate limit.

Why it is a caching decision: This is not caching in the memoisation sense — it is shared ephemeral state. The requirement is an atomic increment and an expiry, across hosts, in well under a millisecond.

# Requirements
# - INCR / EXPIRE atomically, so two instances cannot both admit the 101st request
# - TTL per key
# - sub-millisecond round trip
# - survives an instance restart (not necessarily a server restart)

Answer: redis-py. The data structures are the reason — a counter with an expiry, a set of active sessions, a sorted set of recent attempts — and the survey’s finding that “Redis wins on what is not caching” is this use case.

Use Case 4: Memoising a function in one process#

Who: Any Python developer with a pure-enough function called repeatedly with the same arguments — parsing, a lookup table, a computed property.

Why it is a caching decision: The first reach, and the one most often over-engineered.

# Requirements
# - a decorator
# - keys built from the arguments
# - no dependency, or at most one
# - optionally: a time-to-live, a size in bytes, a way to inspect and clear

Answer: functools.lru_cache until one of three things is true — the entries must expire on a clock, the cache must be bounded by memory rather than count, or a scan-heavy pattern wants a policy other than LRU. Then cachetools, and accept a microsecond per hit, which is nothing against any function worth wrapping. If the function costs under ~10 µs, do not cache it: the decorator is the cost.

Use Case 5: Development and test environments#

Who: A developer who wants production-shaped caching behavior on a laptop and in CI without installing a server.

Why it is a caching decision: The cache’s tier in production is often the network; in development the same code should run against something with no daemon and the same API shape.

Answer: diskcache for a persistent local stand-in; dogpile.cache if the codebase already wants one API over two backends. A developer who can run Valkey locally loses nothing by doing so.

Use Case 6: The cache that legal has to sign off on#

Who: A platform team at a company whose open-source policy bans AGPL outright, or an ISV that ships its product into customer data centers and cannot carry source-available terms (RSALv2 / SSPLv1) on a component. Both are common; neither existed as a caching problem before 2024.

Why it is a caching decision now: Redis 8 is offered under RSALv2 or SSPLv1 or AGPLv3. Each of the three fails a different policy. The team still wants the Redis wire protocol, the redis-py code they already have, and a managed option on their cloud.

# Requirements:
# - OSI-approved permissive license on the server (BSD / Apache / MIT)
# - Existing redis-py code runs unchanged
# - Managed offering on AWS or GCP
# - A way back if the fork stalls

Answer: Valkey (BSD-3-Clause) behind the existing redis-py client. On AWS, ElastiCache for Valkey is also the cheaper SKU (20% node-based, 33% serverless); on GCP, Memorystore for Valkey. Measured behind redis-py, Valkey 9.1.1 and Redis 8.10.1 are the same number (106 and 110 µs median). The “way back” is real only while the data stays on Redis ≤ 7.2 semantics — Valkey reads those RDB files, Redis 7.4+ files it does not, and the reverse move from Valkey 8/9 features (hash-field expiry, cluster databases) is not guaranteed either. Write that down in the decision record.

The mirror persona: a team already on Redis 7.4+ or 8 using Stack modules (JSON, search, vector sets). For them the license question is settled by inertia — leaving means a key-by-key copy over the wire and re-homing modules onto valkey-json / valkey-search, which are younger (1.x, 2025–2026). Staying on Redis under AGPLv3 is usually fine for an internal service that does not distribute the server; it is the ISV case that forces the move.

Use Case 7: The batch job that reads more than fits#

Who: Anyone running a nightly or hourly job that walks a table, a corpus or a key space larger than any cache it could be given, once per pass.

Why it is a caching decision: It looks like the ideal candidate — the same keys, every run — and it is the one case where a cache does nothing. Measured on a loop over 10,000 keys: LRU and FIFO score 0.0% at 1%, 5%, 10%, 25% and 50% of the keys. Each key is evicted one step before it comes round again.

Answer: Size the cache to the working set or do not add one. This is the persona the floor model’s second panel was built to show.

Constraint-Based Decision Matrix#

By tier#

By what a miss costs#

Computed with core.payoff() from the measured hit and store costs (TTLCache 1.2 / 1.4 µs; diskcache 4 / 350 µs; redis-py 106 / 113 µs median). The floor model recomputes it on your machine.

By access pattern (10% of keys cached, 30,000 requests over 10,000 keys)#

Risk Assessment by Requirements#

Conclusion#

No single library meets every need, and the reason is structural: the tiers are different problems. The sequence that fits most teams is functools.lru_cache → cachetools when it needs a clock or a weight → diskcache when it must survive a restart → redis-py when a second machine needs the same answer — each step taken when the previous tier’s readers are not the only readers. And before any of them: know the access pattern, because on a loop the best library in the survey scores nothing.

S4 Strategic Discovery: Caching Libraries#

Date: 2026-08-20. Who maintains each library, what has moved in the category, what is coming, and when each choice stops paying. Maintenance signals are GitHub and PyPI, read 2026-08-20.

Viability, library by library#

Two entrants the survey’s core does not carry but a reader will meet:

• cachebox (6.2.5, MIT, Rust, 427 stars, pushed 2026-08): compiled wheels for CPython 3.10+, measured 0.28 µs per hit — half cachetools, ten times lru_cache. Not in Pyodide. It is what you reach for when cachetools’ microsecond is measurable in your profile, which is rare.
• theine (2.0.0, BSD-3, 429 stars, pushed 2025-11): W-TinyLFU, the scan-resistant admission policy cachetools lacks. Measured elsewhere, not here; its claim is hit rate on mixed traffic, which the floor model’s “hot set plus scan” pattern is the test for.

What moved since 2025#

The server split and re-joined. Redis 7.4 left BSD for RSALv2 / SSPLv1 in March 2024; the Linux Foundation forked 7.2.4 as Valkey the same month; AWS re-priced ElastiCache for Valkey 20–33% below Redis OSS in October 2024; Redis 8 added AGPLv3 in May 2025 and says it intends to keep it. Measured behind the same client, the two servers are the same number. The decision moved from engineering to procurement, and it is one-way past Redis 7.2 because the RDB format diverged. The dates, versions and clients are in S2 §0.5.

RESP3 became the default on the wire (redis-py 8.0, 2026). The visible effect for a cache is none; the invisible one is that client-side caching — the server telling the client which keys it holds have changed — rides on RESP3’s push messages. That is the mechanism by which the network tier can acquire an in-process tier in front of it without the application managing two caches.

The Python clients’ throughput ceiling is the round trip. The figures in circulation — “100,000+ ops/sec” for redis-py, “200,000+” for memcached — are server figures. A single synchronous Python client does 3,000–10,000 round trips a second on a socket (106 µs median here), and 100,000+ only when pipelined (7 µs per key). The measurements are S2 §0.3; the practice is “never fetch two keys in two trips”.

The standard library got cheaper to beat and harder to justify beating. lru_cache is a dict lookup. cachetools’ microsecond buys TTL, weight and policy; cachebox’s 0.28 µs buys a compiled dependency. For almost every function worth caching, neither cost is visible.

What is coming#

• Scan-resistant policies in the mainstream. W-TinyLFU (Caffeine’s policy, theine’s) and ARC solve the pattern on which LRU loses to LFU — a hot set with a scan through it, 49% against 53% at 10% here. cachetools has no such policy and has not announced one; the gap is small on most traffic and decisive on some.
• Compiled in-process caches (cachebox, and the Rust ecosystem generally) will keep narrowing the gap to lru_cache while adding the features it lacks. Expect them to remain optional.
• Server-assisted client caching (RESP3 tracking) as the default two-tier architecture, with the client library managing invalidation. redis-py ships the machinery; the defaults do not turn it on.
• Valkey and Redis diverging in features while staying compatible on the cache path. Valkey 9 added cluster databases, atomic slot migration and hash-field expiry; Redis 8 added vector sets and bundled the Stack modules. GET/SET/SETEX/EXPIRE/pipelines are identical and will stay so — that is the compatibility both projects are paid to keep.
• diskcache’s successor question. Two quiet years on a 37M-download library is not a crisis — SQLite does not move underneath it — but a maintained fork or a replacement built on the same idea is the likely shape of the next five years.

When each choice stops paying#

Risks#

Strategic recommendation#

Adopt in the order of tiers and measure before each step. In-process with the standard library first; cachetools when a clock or a weight is needed; diskcache when the value must survive the process; redis-py when a second machine needs the answer — against Valkey unless the team is already on Redis 7.4+ with Stack modules, in which case staying is cheaper than the key-by-key copy out. Treat the hit rate as a property of the traffic to be measured, not a property of the library to be assumed, and put a number on the miss before adding the cache.

Measured: 2026-08-20