Introduction
Streamlit’s @st.cache_data is per-process — each instance has its own cache. If you run 3 Streamlit instances behind a load balancer, each has a separate cache, and users see inconsistent data. Redis fixes this: a shared, distributed cache that all instances read from.
This post adds Redis to a Streamlit app for caching, session storage, and rate limiting.
Why Redis?
- Shared across instances — all Streamlit processes read the same cache.
- Survives restarts — data persists (unlike in-memory cache).
- Fast — sub-millisecond reads, in-memory.
- Versatile — strings, hashes, lists, sets, sorted sets, pub/sub.
- Free tier — Upstash, Redis Cloud, or self-host.
Setup
pip install redis streamlit
Get a free Redis instance from Upstash (serverless, free tier) or run locally:
docker run -d -p 6379:6379 redis:alpine
Step 1: Connect
import streamlit as st
import redis, json, os
@st.cache_resource
def get_redis():
return redis.from_url(
os.environ.get("REDIS_URL", "redis://localhost:6379"),
decode_responses=True,
)
r = get_redis()
Use @st.cache_resource so the connection is created once per process.
Step 2: Basic caching
import hashlib
def cache_key(func_name, *args):
raw = f"{func_name}:{args}"
return hashlib.md5(raw.encode()).hexdigest()
def cached_fetch(key, ttl=300):
val = r.get(key)
if val is not None:
return json.loads(val)
# Cache miss — fetch and store
data = expensive_api_call()
r.setex(key, ttl, json.dumps(data))
return data
setex sets a value with an expiry (TTL). After TTL, the key auto-deletes.
Step 3: Decorator pattern
import functools
def redis_cache(ttl=300):
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
key = f"{func.__name__}:{args}:{sorted(kwargs.items())}"
cached = r.get(key)
if cached is not None:
return json.loads(cached)
result = func(*args, **kwargs)
r.setex(key, ttl, json.dumps(result))
return result
return wrapper
return decorator
@redis_cache(ttl=60)
def get_user(user_id):
# Expensive DB query
return db.query("SELECT * FROM users WHERE id=?", (user_id,))
Now every Streamlit instance shares the same cache. User A’s request warms the cache; User B reads it instantly.
Step 4: Session storage (persistent login)
Streamlit session state resets on refresh. Use Redis for persistent sessions:
import uuid
def create_session(username):
session_id = str(uuid.uuid4())
r.setex(f"session:{session_id}", 3600, username) # 1 hour TTL
return session_id
def get_session(session_id):
return r.get(f"session:{session_id}")
def destroy_session(session_id):
r.delete(f"session:{session_id}")
# In your app
if "session_id" not in st.session_state:
st.session_state.session_id = None
if not st.session_state.session_id:
# Show login
...
if login_ok:
st.session_state.session_id = create_session(username)
st.rerun()
else:
username = get_session(st.session_state.session_id)
if not username:
st.session_state.session_id = None
st.rerun()
st.write(f"Welcome, {username}")
Step 5: Rate limiting
Protect your app from abuse:
def check_rate_limit(user_id, max_requests=10, window=60):
key = f"ratelimit:{user_id}"
current = r.incr(key)
if current == 1:
r.expire(key, window)
if current > max_requests:
return False
return True
# In your app
if not check_rate_limit(st.session_state.get("username", "anon")):
st.error("Too many requests. Please wait a minute.")
st.stop()
INCR + EXPIRE is the standard Redis rate-limiting pattern — atomic and fast.
Step 6: Pub/Sub for real-time
Broadcast events across instances:
import threading
def listener():
pubsub = r.pubsub()
pubsub.subscribe("notifications")
for message in pubsub.listen():
if message["type"] == "message":
st.toast(message["data"]) # show notification
# Start listener in background thread
if "listener_started" not in st.session_state:
threading.Thread(target=listener, daemon=True).start()
st.session_state.listener_started = True
# Publish from anywhere
r.publish("notifications", "New data available!")
Step 7: Cache invalidation
# Invalidate a specific key
r.delete("get_user:123")
# Invalidate by pattern (careful in production)
for key in r.scan_iter("get_user:*"):
r.delete(key)
# Set a "version" key for bulk invalidation
r.incr("data_version")
key = f"get_user:123:v{r.get('data_version')}"
The version pattern is safer than scanning — increment the version to invalidate all related keys at once.
Common mistakes
- Storing non-serializable objects — Redis stores strings/bytes. Use
json.dumps/json.loadsfor complex objects. - No TTL — keys live forever. Always set an expiry.
- Blocking the event loop — Redis calls are sync. In async apps, use
aioredis. - Connection per request — create one connection per process with
@st.cache_resource. - Storing large objects — Redis is in-memory. Don’t cache 1GB DataFrames; cache aggregates.
Key Concepts
- Redis = in-memory key-value store with TTL
setex= set with expiryincr+expire= atomic rate limiting- Pub/Sub = real-time messaging
- Distributed cache = shared across processes/instances
What to Try Next
- Add Redis to your admin panel for shared session storage.
- Use Redis for rate limiting on your contact form.
- Combine with caching — Redis for cross-instance,
st.cache_datafor per-process. - Deploy multiple Streamlit instances behind a load balancer with shared Redis.
FAQ
Redis vs Streamlit’s built-in cache?
st.cache_data = per-process, in-memory, free. Redis = distributed, persistent, shared. Use built-in for single-instance apps; Redis for multi-instance or when you need persistence.
Upstash vs Redis Cloud vs self-hosted?
Upstash = serverless, pay-per-request, great for low traffic. Redis Cloud = managed, fixed pricing, good for steady traffic. Self-hosted = free but you manage it. Start with Upstash (free tier).
Does Redis work on Streamlit Community Cloud?
Yes — store the Redis URL in Streamlit secrets. Upstash works great with Community Cloud since it’s serverless (no VPC needed).