Introduction
The chat completions API is stateless — you send the full message history every time. The Assistants API is stateful: it manages threads, memory, tool calls, and file search for you. Build once, chat forever.
This post builds a working assistant with tools and file search, in Python.
Core concepts
- Assistant = the bot (model + instructions + tools). Created once.
- Thread = a conversation. Created per user/session.
- Message = a user or assistant message in a thread.
- Run = “process this thread now” — the assistant thinks and replies.
- Tools = functions the assistant can call (code interpreter, file search, custom functions).
Setup
pip install openai
export OPENAI_API_KEY=sk-...
Step 1: Create an assistant
from openai import OpenAI
client = OpenAI()
assistant = client.beta.assistants.create(
name="Data Analyst",
instructions="You analyze CSV data and answer questions. Be concise.",
model="gpt-4o",
tools=[{"type": "code_interpreter"}, {"type": "file_search"}],
)
print(assistant.id) # asst_...
code_interpreter lets the assistant run Python (charts, calculations). file_search lets it search uploaded files (RAG built in).
Step 2: Create a thread and add a message
thread = client.beta.threads.create()
client.beta.threads.messages.create(
thread_id=thread.id,
role="user",
content="What's the average revenue by region?",
)
The thread persists — you can add more messages later and the assistant remembers.
Step 3: Run the assistant
run = client.beta.threads.runs.create_and_poll(
thread_id=thread.id,
assistant_id=assistant.id,
)
if run.status == "completed":
messages = client.beta.threads.messages.list(thread_id=thread.id)
for m in messages.data:
if m.role == "assistant":
print(m.content[0].text.value)
create_and_poll blocks until the run completes (handles tool calls internally). For production, use create + polling/webhooks.
Step 4: Upload a file and attach it
file = client.files.create(
file=open("sales.csv", "rb"),
purpose="assistants",
)
# Attach to thread via message
client.beta.threads.messages.create(
thread_id=thread.id,
role="user",
content="Analyze this data.",
attachments=[{"file_id": file.id, "tools": [{"type": "file_search"}]}],
)
The assistant can now search the CSV and answer questions about it. That’s built-in RAG — no Chroma, no embeddings code.
Step 5: Custom function tools
assistant = client.beta.assistants.create(
name="Weather Bot",
instructions="You help with weather. Use the get_weather tool.",
model="gpt-4o",
tools=[{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}],
)
When the assistant calls the tool, the run pauses with status requires_action. You handle it:
run = client.beta.threads.runs.create(thread_id=thread.id, assistant_id=assistant.id)
while run.status in ("queued", "in_progress", "requires_action"):
if run.status == "requires_action":
tool_outputs = []
for tc in run.required_action.submit_tool_outputs.tool_calls:
city = json.loads(tc.function.arguments)["city"]
result = get_weather(city) # your function
tool_outputs.append({"tool_call_id": tc.id, "output": result})
run = client.beta.threads.runs.submit_tool_outputs_and_poll(
thread_id=thread.id, run_id=run.id, tool_outputs=tool_outputs
)
else:
run = client.beta.threads.runs.retrieve(thread_id=thread.id, run_id=run.id)
This is the same agent loop as from-scratch agents — LLM + tools + loop — but managed by OpenAI.
Step 6: Streaming responses
with client.beta.threads.runs.stream(
thread_id=thread.id,
assistant_id=assistant.id,
) as stream:
for event in stream:
if event.event == "thread.message.delta":
for content in event.data.delta.content:
if content.type == "text":
print(content.text.value, end="", flush=True)
Stream tokens to a Streamlit UI with st.write_stream for a live typing effect.
Assistants API vs Chat Completions
| Chat Completions | Assistants API | |
|---|---|---|
| State | Stateless (you send history) | Stateful (threads persist) |
| Tool calls | You handle the loop | OpenAI handles it |
| File search | You build RAG | Built-in (file_search) |
| Code execution | You run it | Built-in (code_interpreter) |
| Cost | Per token | Per token + storage |
| Complexity | Low | Medium |
| Best for | Simple chat, custom agents | Complex, stateful, file-heavy apps |
Common mistakes
- Not polling correctly —
create_and_pollis easiest; for production, usecreate+ polling with backoff. - Forgetting to handle
requires_action— custom tools pause the run; you must submit outputs or it hangs. - Uploading files without purpose — files need
purpose="assistants"to be usable. - Threads growing forever — old messages accumulate; create new threads per session or trim.
- Hardcoding assistant IDs — store them in env vars or a DB; don’t recreate on every request.
Key Concepts
- Assistant = bot config (model, instructions, tools)
- Thread = conversation (persists across runs)
- Run = process a thread (think + reply)
- Tool = function the assistant can call
- File search = built-in RAG over uploaded files
- Code interpreter = sandboxed Python execution
What to Try Next
- Build a data analysis assistant that answers questions about uploaded CSVs.
- Add custom tools (weather, search, DB queries) via function calling.
- Stream responses to a Streamlit chat UI.
- Compare with MCP — Assistants API tools are server-side; MCP tools are protocol-based.
FAQ
Assistants API vs building your own agent?
Assistants API = faster to ship, managed state, built-in RAG/code exec. Custom agent = full control, cheaper at scale, works with any model. Start with Assistants API for prototypes; build custom for production cost control.
Is the Assistants API expensive?
It’s priced per token (same as chat completions) plus storage for files/threads. For high-volume apps, building custom is cheaper. For low-volume, the dev-time savings are worth it.
Can I use the Assistants API with local models?
No — it’s OpenAI-only. For local models, build a custom agent with Ollama + LangGraph. The patterns (threads, tools, runs) are the same.