Tech With Galvan
Back to Articles
AIJun 14, 2026 • 5 min read

Cloudflare Workers AI: Run LLMs at the Edge (Free Tier)

Run LLMs, embeddings, and image models at the edge with Cloudflare Workers AI — free tier, no GPU, Python/JS examples, and a Streamlit-free deployment.

Galvan
Galvan

Founder & Creator

Introduction

Running LLMs usually means a GPU box, a cloud VM, or paying per token to OpenAI/Anthropic. Cloudflare Workers AI flips that: run Llama, Mistral, embeddings, and image models on Cloudflare’s edge network — with a generous free tier and no infrastructure to manage.

This post shows how to call Workers AI from a Worker (JS) and from Python, plus a real example: an AI summarizer endpoint.

What’s available

Cloudflare Workers AI hosts 50+ models:

  • LLMs: Llama 3.1/3.2/3.3, Mistral, Qwen, Gemma
  • Embeddings: BGE, GTE (for RAG)
  • Image: Stable Diffusion XL, FLUX
  • Speech: Whisper (STT)
  • Translation & classification

All runnable with one API call. Pricing: 10,000 neurons/day free, then pay-per-use (very cheap).

Setup

You need a Cloudflare account (free). Then:

npm create cloudflare@latest my-ai-app
cd my-ai-app

Choose “Hello World” Worker. Then add the AI binding in wrangler.toml:

name = "my-ai-app"
main = "src/index.js"
compatibility_date = "2026-06-20"

[ai]
binding = "AI"

Step 1: Call an LLM from a Worker (JS)

// src/index.js
export default {
  async fetch(request, env) {
    const { prompt } = await request.json();
    const result = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
      messages: [
        { role: "system", content: "You are a helpful assistant. Be concise." },
        { role: "user", content: prompt },
      ],
    });
    return Response.json({ reply: result.response });
  },
};

Deploy:

npx wrangler deploy

Your endpoint is live globally in ~30 seconds. Test:

curl -X POST https://my-ai-app.your-subdomain.workers.dev \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Explain RAG in one sentence."}'

Step 2: Call from Python (REST API)

Workers AI is also a REST API — call it from anywhere:

import requests, os

ACCOUNT_ID = os.environ["CF_ACCOUNT_ID"]
TOKEN = os.environ["CF_API_TOKEN"]

def ask(prompt):
    url = f"https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/meta/llama-3.1-8b-instruct"
    r = requests.post(url, headers={"Authorization": f"Bearer {TOKEN}"},
                      json={"messages": [{"role": "user", "content": prompt}]})
    return r.json()["result"]["response"]

print(ask("What is MCP in one sentence?"))

Get your account ID and API token from the Cloudflare dashboard.

Step 3: Embeddings for RAG

const embeddings = await env.AI.run("@cf/baai/bge-base-en-v1.5", {
  text: ["Hello world", "How are you"],
});
// embeddings.data = [[0.12, -0.04, ...], ...]

Store them in Cloudflare Vectorize (or your own DB) for a fully edge-hosted RAG pipeline.

Step 4: Image generation

const image = await env.AI.run("@cf/stabilityai/stable-diffusion-xl-base-1.0", {
  prompt: "A futuristic city at sunset, cyberpunk style",
});
// image is a ReadableStream of PNG bytes
return new Response(image, { headers: { "Content-Type": "image/png" } });

Step 5: Real example — AI summarizer endpoint

A complete Worker that summarizes any text:

export default {
  async fetch(request, env) {
    if (request.method !== "POST") {
      return new Response("POST a JSON body with { text }", { status: 405 });
    }
    const { text } = await request.json();
    if (!text || text.length > 10000) {
      return Response.json({ error: "text required, max 10k chars" }, { status: 400 });
    }
    const result = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
      messages: [
        { role: "system", content: "Summarize the following text in 3 bullet points." },
        { role: "user", content: text },
      ],
    });
    return Response.json({ summary: result.response });
  },
};

Workers AI vs alternatives

Workers AI OpenAI API Ollama (local)
Cost Free tier + cheap Per-token Free
Latency Edge (low) Cloud Local (depends)
Privacy Data to Cloudflare Data to OpenAI 100% local
Model quality Good (8B-70B) Best Depends on hardware
Setup wrangler deploy API key Install + pull
Scaling Automatic Automatic Manual

Common mistakes

  • Using huge models on the free tier — 70B models burn neurons fast. Start with 8B.
  • No input validation — always cap input length; LLMs can be expensive on long inputs.
  • Forgetting CORS — if calling from a browser, add CORS headers.
  • Storing the API token client-side — keep it in the Worker (server-side), never in frontend code.
  • Ignoring cold starts — first request can be ~1-2s; subsequent are fast.

Key Concepts

  • Workers AI = LLMs/embeddings/images at the edge
  • Binding = env.AI in your Worker
  • Neurons = Cloudflare’s compute unit (free tier: 10k/day)
  • REST API = call from any language
  • Edge deployment = global, low-latency

What to Try Next

  • Build a RAG pipeline with Workers AI embeddings + Vectorize.
  • Add an AI endpoint to your Astro site (this site runs on Cloudflare Pages).
  • Combine with MCP — expose a Worker as an MCP tool.
  • Build a Streamlit app that calls Workers AI as the backend.

FAQ

Is the free tier enough for a side project?

Yes — 10,000 neurons/day is plenty for a personal tool or low-traffic API. A typical Llama 8B request costs ~100-500 neurons.

Workers AI vs Hugging Face Inference API?

HF Inference API has more models but rate limits and cold starts. Workers AI is faster (edge) and integrated with Cloudflare’s ecosystem (Pages, Workers, Vectorize). Pick Workers AI if you’re already on Cloudflare.

Can I fine-tune models on Workers AI?

Not directly. Workers AI runs pre-trained models. For fine-tuning, use a separate platform (HF, Together, etc.) and serve via Workers AI’s custom model support (limited). For most use cases, prompt engineering is enough.