Tech With Galvan
Back to Articles
AIJul 12, 2027 • 6 min read

Streamlit + LangChain: Chatbot with Memory and Tools

Build an advanced LangChain chatbot in Streamlit — conversation memory, tool calling, streaming, and source citations. Production-ready patterns.

Galvan
Galvan

Founder & Creator

Introduction

The basic RAG chatbot answers questions about documents. But a production chatbot needs more: conversation memory (remember previous turns), tool calling (search, calculate, fetch data), and streaming (token-by-token). This post builds all three with LangChain + Streamlit.

What you’ll build

  • Streaming responses with st.write_stream
  • Conversation memory (buffer + summary)
  • Tool calling (weather, calculator, search)
  • Source citations
  • Clean chat UI with avatars

Setup

pip install streamlit langchain langchain-openai langchain-community

Step 1: Tools

from langchain.tools import tool
import requests, math

@tool
def get_weather(city: str) -> str:
    """Get current weather for a city."""
    # Replace with real API
    return f"22°C, sunny in {city}"

@tool
def calculate(expression: str) -> str:
    """Evaluate a math expression safely."""
    try:
        return str(eval(expression, {"__builtins__": {}}, {"math": math}))
    except Exception as e:
        return f"Error: {e}"

@tool
def search_web(query: str) -> str:
    """Search the web for information."""
    # Replace with real search API (DuckDuckGo, Tavily, etc.)
    return f"Search results for: {query}"

tools = [get_weather, calculate, search_web]

Step 2: LLM with tools

from langchain_openai import ChatOpenAI
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0, streaming=True)

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant. Use tools when needed. Be concise."),
    ("placeholder", "{chat_history}"),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}"),
])

agent = create_tool_calling_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

Step 3: Conversation memory

from langchain.memory import ConversationBufferWindowMemory

memory = ConversationBufferWindowMemory(
    k=10,                        # keep last 10 turns
    memory_key="chat_history",
    return_messages=True,
    output_key="output",
)

ConversationBufferWindowMemory keeps the last K turns. For long conversations, use ConversationSummaryMemory to compress older turns.

Step 4: Streaming with callbacks

from langchain.callbacks.base import BaseCallbackHandler
from queue import Queue
import threading

class StreamHandler(BaseCallbackHandler):
    def __init__(self, queue):
        self.queue = queue

    def on_llm_new_token(self, token, **kwargs):
        self.queue.put(token)

    def on_llm_end(self, *args, **kwargs):
        self.queue.put(None)   # sentinel

def stream_response(executor, memory, user_input):
    queue = Queue()
    handler = StreamHandler(queue)

    def run():
        result = executor.invoke(
            {"input": user_input, "chat_history": memory.chat_memory.messages},
            config={"callbacks": [handler]},
        )
        memory.save_context({"input": user_input}, {"output": result["output"]})
        queue.put(None)

    thread = threading.Thread(target=run)
    thread.start()

    while True:
        token = queue.get()
        if token is None:
            break
        yield token

Step 5: Streamlit UI

import streamlit as st

st.title("🤖 LangChain Chatbot")
st.caption("Memory + Tools + Streaming")

if "messages" not in st.session_state:
    st.session_state.messages = []
if "memory" not in st.session_state:
    st.session_state.memory = ConversationBufferWindowMemory(
        k=10, memory_key="chat_history", return_messages=True, output_key="output"
    )

for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        st.write(msg["content"])

if prompt := st.chat_input("Ask anything..."):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.write(prompt)

    with st.chat_message("assistant"):
        response = st.write_stream(stream_response(executor, st.session_state.memory, prompt))
    st.session_state.messages.append({"role": "assistant", "content": response})

Step 6: Tool usage display

Show which tools were called:

with st.chat_message("assistant"):
    with st.status("Thinking...", expanded=True) as status:
        st.write("Calling tools...")
        response = stream_response(executor, memory, prompt)
        full = st.write_stream(response)
        status.update(label="Done", state="complete", expanded=False)
    st.write(full)

Step 7: Clear memory

with st.sidebar:
    if st.button("Clear chat"):
        st.session_state.messages = []
        st.session_state.memory.clear()
        st.rerun()

Step 8: Persist memory (database)

import sqlite3

def save_message(session_id, role, content):
    conn = sqlite3.connect("chat.db")
    conn.execute("INSERT INTO messages (session_id, role, content) VALUES (?,?,?)",
                 (session_id, role, content))
    conn.commit()
    conn.close()

def load_history(session_id):
    conn = sqlite3.connect("chat.db")
    rows = conn.execute("SELECT role, content FROM messages WHERE session_id=? ORDER BY id", (session_id,)).fetchall()
    conn.close()
    return [{"role": r[0], "content": r[1]} for r in rows]

Common mistakes

  • No memory — every turn is stateless. Add ConversationBufferWindowMemory.
  • Blocking streaming — executor.invoke blocks. Use threading + queue for true streaming.
  • Tool descriptions too vague — the model won’t know when to use them. Write clear docstrings.
  • Unbounded history — context overflow. Use window memory (k=10) or summary memory.
  • No error handling — tool failures crash the agent. Wrap tools in try/except.

Key Concepts

  • Agent = LLM + tools + prompt
  • Tool calling = model decides when to call functions
  • Memory = conversation history passed to the LLM
  • Streaming = token-by-token output via callbacks
  • AgentExecutor = runs the agent loop (think → tool → think → answer)

What to Try Next

  • Add MCP tools for external capabilities.
  • Persist memory with Supabase.
  • Add RAG — combine tools with document retrieval.
  • Deploy with Docker.

FAQ

LangChain agent vs OpenAI Assistants API?

LangChain = you control the loop, works with any model, more flexible. Assistants API = managed, less code, OpenAI-only. Use LangChain for control; Assistants API for speed.

How do I reduce latency?

Use gpt-4o-mini (fast), limit memory to k=5, and stream. For tool-heavy agents, the bottleneck is tool execution — parallelize independent tools.

Can I use local models?

Yes — swap ChatOpenAI for ChatOllama. Tool calling quality depends on the model; use 8B+ models for reliable tool use.