Introduction
This project is the classic machine-learning rite of passage, upgraded: instead of scoring 98% on MNIST in a notebook and stopping, you’ll deploy the model behind a drawing canvas. Draw a digit with your mouse, and a trained neural network guesses what you wrote — live. It is the drawing-board sibling of the image classifier, but with a model you train yourself in five minutes.
Two lessons in one: training a small CNN end-to-end, and the surprisingly tricky preprocessing that stands between a canvas drawing and a correct prediction.
Features
- Draw pad — mouse/touch drawing via
streamlit-drawable-canvas. - Trained CNN — 99%+ accuracy on MNIST, trained in ~2 minutes on CPU.
- Live prediction — top-3 guesses with confidence bars.
- Probability chart — all ten digits’ scores at once.
- Model persistence — train once, save to disk, load forever.
Prerequisites
- Python 3.9+ — from python.org.
- Dependencies:
pip install streamlit streamlit-drawable-canvas tensorflow-cpu numpyStep 1: Train the Model (once)
Save as train_model.py and run it:
import tensorflow as tf
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train = x_train.reshape(-1, 28, 28, 1) / 255.0
x_test = x_test.reshape(-1, 28, 28, 1) / 255.0
model = tf.keras.Sequential([
tf.keras.layers.Conv2D(32, 3, activation="relu", input_shape=(28, 28, 1)),
tf.keras.layers.MaxPooling2D(),
tf.keras.layers.Conv2D(64, 3, activation="relu"),
tf.keras.layers.MaxPooling2D(),
tf.keras.layers.Flatten(),
tf.keras.layers.Dropout(0.3),
tf.keras.layers.Dense(10, activation="softmax"),
])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"])
model.fit(x_train, y_train, epochs=5, validation_split=0.1)
print("Test accuracy:", model.evaluate(x_test, y_test)[1])
model.save("digit_model.keras")Step 2: Create the App
Save as digit_app.py:
import streamlit as st
import numpy as np
from PIL import Image, ImageOps, ImageDraw
from streamlit_drawable_canvas import st_canvas
from tensorflow.keras.models import load_model
st.set_page_config(page_title="Digit Recognizer", page_icon="✍️")
st.title("✍️ Draw a digit (0–9)")
model = load_model("digit_model.keras")
SIZE = 280
canvas = st_canvas(
fill_color="black",
stroke_width=20,
stroke_color="white",
background_color="black",
height=SIZE, width=SIZE,
drawing_mode="freedraw",
key="pad",
)
if st.button("🔮 Predict") and canvas.image_data is not None:
img = Image.fromarray(canvas.image_data.astype(np.uint8))
gray = ImageOps.grayscale(img).resize((28, 28))
arr = np.array(gray, dtype=np.float32) / 255.0
if arr.sum() < 1:
st.warning("Draw something first!")
st.stop()
probs = model.predict(arr.reshape(1, 28, 28, 1), verbose=0)[0]
top3 = probs.argsort()[-3:][::-1]
st.subheader(f"Prediction: **{top3[0]}** ({probs[top3[0]]:.0%} confident)")
for digit in top3:
st.progress(float(probs[digit]), text=f"Digit {digit}: {probs[digit]:.1%}")
st.bar_chart({str(d): float(probs[d]) for d in range(10)})Step 3: Run the App
python train_model.py
streamlit run digit_app.pyDraw a big, centered 7 and predict. Then draw a tiny corner slash and watch the model get humbled — that contrast is the preprocessing lesson.
How It Works
Training is the textbook CNN recipe: two convolution + pooling blocks learn to detect strokes and shapes at increasing abstraction, a dropout layer fights overfitting, and a 10-unit softmax outputs one probability per digit. Five epochs on CPU gets ~99% test accuracy because MNIST is famously friendly.
The app’s real lesson is preprocessing symmetry. MNIST digits are: white-on-black, 28×28, centered, thick strokes. Your drawing must match that contract exactly — grayscale, downscale, normalize to 0–1. Get one detail wrong (black-on-white is the classic) and a 99% model reads your confident 5 as an 8. The canvas is configured black-background, white-stroke deliberately, to match the training data without inversion gymnastics.
probs.argsort()[-3:][::-1] is the numpy top-3 idiom: argsort gives ascending indices, slice the last three, reverse for descending — the same ranking idea as the image classifier’s decode_predictions, done by hand.
Common Errors & Fixes
- Predictions are terrible despite 99% test accuracy — preprocessing mismatch. Verify: white strokes on black, values 0–1, shape
(1, 28, 28, 1). Compare your array’s mean against a real MNIST sample. OSError: digit_model.keras does not exist— you skipped Step 1; the app loads a saved model, it doesn’t train one.- Canvas is blank in the prediction —
canvas.image_dataholds the RGBA array; make sure you read it after drawing, and thatstroke_colorcontrastsbackground_color. - Everything predicts as 1 or 0 — your strokes are too thin;
stroke_width=20at 280px scales down to MNIST-like thickness.
Key Concepts
- CNN building blocks — conv filters, pooling, dropout, softmax.
- Train/deploy split — train once offline, serve forever.
- Preprocessing symmetry — inference inputs must mirror training inputs exactly.
- Softmax probabilities — ranked confidence, not just an answer.
What to Try Next
- Add auto-predict on release using the canvas’s
update_streamlitreal-time mode. - Add a center-of-mass recenter step so off-center drawings work better.
- Train on EMNIST letters and recognize A–Z with the same app (26 outputs).
- Compare against the pretrained MobileNet classifier — custom-trained vs off-the-shelf.
FAQ
Why does my hand-drawn digit fail when test accuracy is 99%?
The model is 99% accurate on MNIST-like inputs. Real drawings differ in stroke width, position, and style — the distribution shift eats the margin. Better preprocessing recovers most of it.
Can I skip training and download a model?
Plenty of pretrained MNIST models exist, but training takes two minutes and teaches the whole pipeline. Do it once.
Is 5 epochs enough?
For MNIST with a CNN, yes — accuracy plateaus around 99%. More epochs mostly overfit; watch the validation split to see it happen.