Tech With Galvan
Back to Articles
AIMay 2, 2027 • 5 min read

Streamlit + OpenAI: Chat with GPT-4 (Full App)

Build a production-ready ChatGPT-style app in Streamlit with OpenAI's GPT-4 — streaming, system prompts, temperature control, and cost tracking.

Galvan
Galvan

Founder & Creator

Introduction

ChatGPT is great, but building your own GPT-4 app gives you control: custom system prompts, temperature tuning, cost tracking, and a branded UI. This post builds a production-ready ChatGPT-style app in Streamlit with OpenAI’s API.

Setup

pip install streamlit openai tiktoken

Store your API key in .streamlit/secrets.toml:

OPENAI_API_KEY = "sk-..."

Step 1: Basic chat

import streamlit as st
from openai import OpenAI

client = OpenAI(api_key=st.secrets["OPENAI_API_KEY"])

st.title("💬 Chat with GPT-4")

if "messages" not in st.session_state:
    st.session_state.messages = []

for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        st.write(msg["content"])

if prompt := st.chat_input("Ask anything..."):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.write(prompt)

    with st.chat_message("assistant"):
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=st.session_state.messages,
        )
        reply = response.choices[0].message.content
        st.write(reply)

    st.session_state.messages.append({"role": "assistant", "content": reply})

Step 2: Streaming responses

def stream_response(messages, model, temperature):
    stream = client.chat.completions.create(
        model=model,
        messages=messages,
        temperature=temperature,
        stream=True,
    )
    for chunk in stream:
        if chunk.choices[0].delta.content:
            yield chunk.choices[0].delta.content

# In the UI:
with st.chat_message("assistant"):
    reply = st.write_stream(stream_response(st.session_state.messages, model, temp))
st.session_state.messages.append({"role": "assistant", "content": reply})

st.write_stream renders tokens as they arrive — the “typing” effect.

Step 3: Sidebar controls

with st.sidebar:
    st.header("Settings")
    model = st.selectbox("Model", ["gpt-4o", "gpt-4o-mini", "gpt-4-turbo", "gpt-3.5-turbo"])
    temperature = st.slider("Temperature", 0.0, 2.0, 0.7, 0.1)
    max_tokens = st.slider("Max tokens", 100, 4096, 1024)
    system_prompt = st.text_area("System prompt", value="You are a helpful assistant.")

    if st.button("Clear chat"):
        st.session_state.messages = []
        st.rerun()

    if st.button("Export chat"):
        import json
        st.download_button("Download JSON", data=json.dumps(st.session_state.messages),
                          file_name="chat.json", mime="application/json")

Step 4: System prompt

# Prepend system message to every request
messages = [{"role": "system", "content": system_prompt}] + st.session_state.messages

response = client.chat.completions.create(
    model=model,
    messages=messages,
    temperature=temperature,
    max_tokens=max_tokens,
)

The system prompt shapes the assistant’s behavior. Iterate on it based on user feedback.

Step 5: Cost tracking

import tiktoken

def count_tokens(messages, model="gpt-4o"):
    enc = tiktoken.encoding_for_model(model)
    total = 0
    for m in messages:
        total += len(enc.encode(m["content"])) + 4  # 4 tokens per message overhead
    return total

# Pricing (per 1M tokens, as of 2026)
PRICES = {
    "gpt-4o": {"input": 2.50, "output": 10.00},
    "gpt-4o-mini": {"input": 0.15, "output": 0.60},
    "gpt-4-turbo": {"input": 10.00, "output": 30.00},
}

input_tokens = count_tokens(st.session_state.messages)
# Estimate output tokens (rough)
output_tokens = len(st.session_state.messages) * 50  # rough estimate

cost = (input_tokens * PRICES[model]["input"] + output_tokens * PRICES[model]["output"]) / 1_000_000
st.sidebar.metric("Est. cost", f"${cost:.4f}")

Step 6: Error handling

from openai import RateLimitError, APIError, AuthenticationError

def safe_chat(messages, model, temperature, max_tokens):
    try:
        stream = client.chat.completions.create(
            model=model, messages=messages, temperature=temperature,
            max_tokens=max_tokens, stream=True,
        )
        for chunk in stream:
            if chunk.choices[0].delta.content:
                yield chunk.choices[0].delta.content
    except RateLimitError:
        yield "Rate limit exceeded. Please wait a moment and try again."
    except AuthenticationError:
        yield "Invalid API key. Check your secrets."
    except APIError as e:
        yield f"API error: {e.message}"
    except Exception as e:
        yield f"Unexpected error: {str(e)}"

Step 7: Conversation memory (trim)

def trim_messages(messages, max_tokens=6000):
    """Keep system prompt + recent messages within token budget."""
    system = [m for m in messages if m["role"] == "system"]
    others = [m for m in messages if m["role"] != "system"]

    # Keep last N messages that fit
    kept = []
    total = 0
    for m in reversed(others):
        tokens = count_tokens([m])
        if total + tokens > max_tokens:
            break
        kept.insert(0, m)
        total += tokens
    return system + kept

Step 8: Save/load conversations

import json, os

CONV_DIR = "conversations"
os.makedirs(CONV_DIR, exist_ok=True)

def save_conversation(name):
    with open(f"{CONV_DIR}/{name}.json", "w") as f:
        json.dump(st.session_state.messages, f)

def load_conversation(name):
    with open(f"{CONV_DIR}/{name}.json") as f:
        st.session_state.messages = json.load(f)

# UI
convs = [f.replace(".json", "") for f in os.listdir(CONV_DIR) if f.endswith(".json")]
selected = st.sidebar.selectbox("Load conversation", ["New"] + convs)
if selected != "New" and st.sidebar.button("Load"):
    load_conversation(selected)
    st.rerun()

name = st.sidebar.text_input("Save as")
if name and st.sidebar.button("Save"):
    save_conversation(name)
    st.sidebar.success(f"Saved as {name}")

Step 9: Add tools (function calling)

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"],
            },
        },
    },
]

def chat_with_tools(messages, model, temperature):
    response = client.chat.completions.create(
        model=model, messages=messages, tools=TOOLS, temperature=temperature
    )
    msg = response.choices[0].message

    if msg.tool_calls:
        messages.append(msg)
        for tc in msg.tool_calls:
            import json
            args = json.loads(tc.function.arguments)
            result = get_weather(**args)   # your function
            messages.append({"role": "tool", "tool_call_id": tc.id, "content": result})

        # Get final answer
        final = client.chat.completions.create(model=model, messages=messages, temperature=temperature)
        return final.choices[0].message.content
    return msg.content

Common mistakes

  • No streaming — users stare at a spinner. Always stream.
  • No system prompt — the model’s behavior is unpredictable.
  • Unbounded history — context overflow. Trim regularly.
  • No error handling — API calls fail. Add retry with backoff.
  • Hardcoding the model — let users choose (cost vs quality).
  • No cost tracking — GPT-4 is expensive. Track and display usage.

Key Concepts

  • OpenAI API = chat completions with streaming
  • System prompt = shapes assistant behavior
  • Temperature = creativity dial (0 = deterministic, 2 = creative)
  • Token counting = cost estimation with tiktoken
  • Memory trimming = stay within context window
  • Function calling = model invokes tools

What to Try Next

  • Add RAG — chat with your documents.
  • Add auth for per-user chat history.
  • Persist conversations with Supabase.
  • Deploy with Docker.

FAQ

GPT-4o vs GPT-4o-mini vs GPT-3.5?

GPT-4o = best quality, multimodal, moderate cost. GPT-4o-mini = 10x cheaper, great for most tasks. GPT-3.5 = cheapest, older, less capable. Start with gpt-4o-mini; upgrade to gpt-4o when quality matters.

How do I reduce costs?

Use gpt-4o-mini (10x cheaper), trim conversation history, set max_tokens lower, cache common responses, and use streaming (doesn’t reduce cost but improves UX).

Can I use this with local models?

Yes — swap the OpenAI client for Ollama (OpenAI-compatible API). Change the base URL to http://localhost:11434/v1 and the model to llama3.2. Everything else stays the same. See RAG with local LLMs.