Introduction
Whisper is OpenAI’s open-source speech recognition model — and it’s free to run locally. This post builds a Streamlit app that transcribes audio files, shows timestamps, and lets you download the text. No API keys, no cloud, no cost.
Setup
pip install streamlit openai-whisper
# Also need ffmpeg
brew install ffmpeg # macOS
# sudo apt install ffmpeg # Linux
Step 1: Model selection
import streamlit as st
import whisper
MODEL_SIZES = {
"tiny": "Fastest, least accurate (39M params)",
"base": "Good balance (74M params)",
"small": "Better accuracy (244M params)",
"medium": "High accuracy (769M params)",
"large": "Best accuracy (1550M params)",
}
model_size = st.selectbox("Model", list(MODEL_SIZES.keys()),
format_func=lambda x: f"{x} — {MODEL_SIZES[x]}")
Recommendation: base for most use cases (1s per minute of audio on a modern laptop). small if accuracy matters. large only with a GPU.
Step 2: Load model (cached)
@st.cache_resource
def load_model(size):
return whisper.load_model(size)
model = load_model(model_size)
@st.cache_resource ensures the model loads once, not on every rerun. First load takes 5-30s depending on size.
Step 3: Upload and transcribe
uploaded = st.file_uploader("Upload audio", type=["mp3", "wav", "m4a", "ogg", "flac"])
if uploaded:
# Save to temp file (Whisper needs a file path)
import tempfile, os
with tempfile.NamedTemporaryFile(delete=False, suffix=f".{uploaded.name.split('.')[-1]}") as f:
f.write(uploaded.read())
audio_path = f.name
with st.spinner("Transcribing... (this may take a while)"):
result = model.transcribe(audio_path, verbose=False)
st.subheader("Transcription")
st.write(result["text"])
# Cleanup
os.unlink(audio_path)
Step 4: Show timestamps (segments)
if uploaded:
result = model.transcribe(audio_path)
st.subheader("Segments with timestamps")
for seg in result["segments"]:
start = seg["start"]
end = seg["end"]
text = seg["text"].strip()
st.write(f"**[{start:.1f}s - {end:.1f}s]** {text}")
Each segment has start, end, and text. Great for subtitles or navigation.
Step 5: Language detection
result = model.transcribe(audio_path)
detected_lang = result["language"]
st.write(f"Detected language: **{detected_lang}**")
Whisper auto-detects the language. For better accuracy, specify it:
result = model.transcribe(audio_path, language="en")
Step 6: Translate to English
Whisper can translate any language to English:
result = model.transcribe(audio_path, task="translate")
st.write(result["text"]) # English translation
Step 7: Download transcription
transcript = result["text"]
# Plain text
st.download_button("📥 Download TXT", data=transcript, file_name="transcript.txt")
# SRT subtitles
def to_srt(segments):
srt = ""
for i, seg in enumerate(segments, 1):
start = format_time(seg["start"])
end = format_time(seg["end"])
srt += f"{i}\n{start} --> {end}\n{seg['text'].strip()}\n\n"
return srt
def format_time(seconds):
h = int(seconds // 3600)
m = int((seconds % 3600) // 60)
s = int(seconds % 60)
ms = int((seconds - int(seconds)) * 1000)
return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"
srt = to_srt(result["segments"])
st.download_button("📥 Download SRT", data=srt, file_name="transcript.srt")
Step 8: Word-level timestamps
result = model.transcribe(audio_path, word_timestamps=True)
for seg in result["segments"]:
for word in seg["words"]:
st.write(f"[{word['start']:.2f}] {word['word']} ({word['probability']:.2f})")
Word-level timestamps are useful for precise subtitle sync and audio editing.
Step 9: Real-time recording (browser)
For live recording, use audio-recorder-streamlit:
pip install audio-recorder-streamlit
from audio_recorder_streamlit import audio_recorder
audio_bytes = audio_recorder()
if audio_bytes:
with open("recorded.wav", "wb") as f:
f.write(audio_bytes)
result = model.transcribe("recorded.wav")
st.write(result["text"])
Record in the browser, transcribe instantly. Great for voice notes.
Performance tips
- Use the right model size —
baseis the sweet spot for most use cases. - Cache the model —
@st.cache_resourceis essential. - Process in chunks — for long audio (>10 min), split into chunks.
- Use GPU — 10x faster on a GPU. Set
device="cuda"inwhisper.load_model. - Batch processing — transcribe multiple files in a loop, show progress.
Common mistakes
- Forgetting ffmpeg — Whisper needs it for audio decoding. Install before running.
- Loading model on every rerun — without
@st.cache_resource, the app is unusable. - Using
largeon CPU — takes minutes per minute of audio. Usebaseorsmall. - Not cleaning up temp files — delete after transcription.
- Ignoring language — specifying the language improves accuracy significantly.
Key Concepts
- Whisper = OpenAI’s open-source STT model
- Model sizes = tiny (39M) to large (1.5B) params
- Segments = timestamped chunks of transcription
- Word timestamps = per-word timing for precise sync
- SRT = subtitle file format
What to Try Next
- Build a voice agent — transcribe + LLM + TTS.
- Add translation — transcribe foreign audio to English.
- Combine with sentiment analysis — transcribe + analyze.
- Store transcriptions in Supabase.
FAQ
Whisper vs Google Speech-to-Text vs AWS Transcribe?
Whisper = free, local, open-source, great accuracy. Google/AWS = managed, pay-per-minute, slightly better on noisy audio. For most use cases, Whisper is the best choice (free + private).
Can Whisper handle multiple speakers?
Not natively — it transcribes all speech as one stream. For speaker diarization, use whisperx or pyannote-audio alongside Whisper.
How accurate is Whisper?
On clean audio, near-human accuracy. On noisy audio, accents, or technical jargon, it degrades. Use medium or large for better accuracy. Always review transcriptions for critical content.