Introduction
Running LLMs usually means a GPU box, a cloud VM, or paying per token to OpenAI/Anthropic. Cloudflare Workers AI flips that: run Llama, Mistral, embeddings, and image models on Cloudflare’s edge network — with a generous free tier and no infrastructure to manage.
This post shows how to call Workers AI from a Worker (JS) and from Python, plus a real example: an AI summarizer endpoint.
What’s available
Cloudflare Workers AI hosts 50+ models:
- LLMs: Llama 3.1/3.2/3.3, Mistral, Qwen, Gemma
- Embeddings: BGE, GTE (for RAG)
- Image: Stable Diffusion XL, FLUX
- Speech: Whisper (STT)
- Translation & classification
All runnable with one API call. Pricing: 10,000 neurons/day free, then pay-per-use (very cheap).
Setup
You need a Cloudflare account (free). Then:
npm create cloudflare@latest my-ai-app
cd my-ai-app
Choose “Hello World” Worker. Then add the AI binding in wrangler.toml:
name = "my-ai-app"
main = "src/index.js"
compatibility_date = "2026-06-20"
[ai]
binding = "AI"
Step 1: Call an LLM from a Worker (JS)
// src/index.js
export default {
async fetch(request, env) {
const { prompt } = await request.json();
const result = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
messages: [
{ role: "system", content: "You are a helpful assistant. Be concise." },
{ role: "user", content: prompt },
],
});
return Response.json({ reply: result.response });
},
};
Deploy:
npx wrangler deploy
Your endpoint is live globally in ~30 seconds. Test:
curl -X POST https://my-ai-app.your-subdomain.workers.dev \
-H "Content-Type: application/json" \
-d '{"prompt": "Explain RAG in one sentence."}'
Step 2: Call from Python (REST API)
Workers AI is also a REST API — call it from anywhere:
import requests, os
ACCOUNT_ID = os.environ["CF_ACCOUNT_ID"]
TOKEN = os.environ["CF_API_TOKEN"]
def ask(prompt):
url = f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-3.1-8b-instruct"
r = requests.post(url, headers={"Authorization": f"Bearer {TOKEN}"},
json={"messages": [{"role": "user", "content": prompt}]})
return r.json()["result"]["response"]
print(ask("What is MCP in one sentence?"))
Get your account ID and API token from the Cloudflare dashboard.
Step 3: Embeddings for RAG
const embeddings = await env.AI.run("@cf/baai/bge-base-en-v1.5", {
text: ["Hello world", "How are you"],
});
// embeddings.data = [[0.12, -0.04, ...], ...]
Store them in Cloudflare Vectorize (or your own DB) for a fully edge-hosted RAG pipeline.
Step 4: Image generation
const image = await env.AI.run("@cf/stabilityai/stable-diffusion-xl-base-1.0", {
prompt: "A futuristic city at sunset, cyberpunk style",
});
// image is a ReadableStream of PNG bytes
return new Response(image, { headers: { "Content-Type": "image/png" } });
Step 5: Real example — AI summarizer endpoint
A complete Worker that summarizes any text:
export default {
async fetch(request, env) {
if (request.method !== "POST") {
return new Response("POST a JSON body with { text }", { status: 405 });
}
const { text } = await request.json();
if (!text || text.length > 10000) {
return Response.json({ error: "text required, max 10k chars" }, { status: 400 });
}
const result = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
messages: [
{ role: "system", content: "Summarize the following text in 3 bullet points." },
{ role: "user", content: text },
],
});
return Response.json({ summary: result.response });
},
};
Workers AI vs alternatives
| Workers AI | OpenAI API | Ollama (local) | |
|---|---|---|---|
| Cost | Free tier + cheap | Per-token | Free |
| Latency | Edge (low) | Cloud | Local (depends) |
| Privacy | Data to Cloudflare | Data to OpenAI | 100% local |
| Model quality | Good (8B-70B) | Best | Depends on hardware |
| Setup | wrangler deploy |
API key | Install + pull |
| Scaling | Automatic | Automatic | Manual |
Common mistakes
- Using huge models on the free tier — 70B models burn neurons fast. Start with 8B.
- No input validation — always cap input length; LLMs can be expensive on long inputs.
- Forgetting CORS — if calling from a browser, add CORS headers.
- Storing the API token client-side — keep it in the Worker (server-side), never in frontend code.
- Ignoring cold starts — first request can be ~1-2s; subsequent are fast.
Key Concepts
- Workers AI = LLMs/embeddings/images at the edge
- Binding =
env.AIin your Worker - Neurons = Cloudflare’s compute unit (free tier: 10k/day)
- REST API = call from any language
- Edge deployment = global, low-latency
What to Try Next
- Build a RAG pipeline with Workers AI embeddings + Vectorize.
- Add an AI endpoint to your Astro site (this site runs on Cloudflare Pages).
- Combine with MCP — expose a Worker as an MCP tool.
- Build a Streamlit app that calls Workers AI as the backend.
FAQ
Is the free tier enough for a side project?
Yes — 10,000 neurons/day is plenty for a personal tool or low-traffic API. A typical Llama 8B request costs ~100-500 neurons.
Workers AI vs Hugging Face Inference API?
HF Inference API has more models but rate limits and cold starts. Workers AI is faster (edge) and integrated with Cloudflare’s ecosystem (Pages, Workers, Vectorize). Pick Workers AI if you’re already on Cloudflare.
Can I fine-tune models on Workers AI?
Not directly. Workers AI runs pre-trained models. For fine-tuning, use a separate platform (HF, Together, etc.) and serve via Workers AI’s custom model support (limited). For most use cases, prompt engineering is enough.