Tech With Galvan
Back to Articles
AIJan 17, 2027 • 6 min read

Build a RAG Chatbot with Streamlit + LangChain (Chat with Your Docs)

Build a RAG chatbot in Streamlit with LangChain — load documents, embed, retrieve, and chat with your own data. With streaming and source citations.

Galvan
Galvan

Founder & Creator

Introduction

Chat with PDF used a cloud API with manual chunking. LangChain makes RAG easier: document loaders, text splitters, vector stores, and retrieval chains are all built in. This post builds a RAG chatbot in Streamlit with LangChain — load docs, embed, retrieve, and chat, with streaming and source citations.

What you’ll build

  • Upload PDFs/TXT/MD
  • Auto-chunk and embed with OpenAI (or local Ollama)
  • Chat with your documents
  • Stream responses with source citations

Setup

pip install streamlit langchain langchain-openai langchain-community chromadb pypdf

Set OPENAI_API_KEY in env or .streamlit/secrets.toml.

Step 1: Load and chunk documents

from langchain_community.document_loaders import PyPDFLoader, TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter

def load_and_split(uploaded_file):
    # Save to temp file
    import tempfile, os
    with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as f:
        f.write(uploaded_file.read())
        path = f.name

    loader = PyPDFLoader(path)
    docs = loader.load()

    splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=200,
    )
    return splitter.split_documents(docs)

RecursiveCharacterTextSplitter tries to split on paragraphs, then sentences, then words — keeping chunks coherent.

Step 2: Create vector store

from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma

def create_vectorstore(chunks):
    embeddings = OpenAIEmbeddings()
    return Chroma.from_documents(chunks, embeddings)

For local/private (no API): use OllamaEmbeddings + local Chroma. See RAG with local LLMs.

Step 3: Create retrieval chain

from langchain_openai import ChatOpenAI
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate

def create_chain(vectorstore):
    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
    retriever = vectorstore.as_retriever(search_kwargs={"k": 4})

    prompt = ChatPromptTemplate.from_template("""
    Answer the question based only on the following context.
    If you don't know, say "I don't know based on the provided documents."

    Context: {context}
    Question: {input}
    """)

    document_chain = create_stuff_documents_chain(llm, prompt)
    return create_retrieval_chain(retriever, document_chain)

Step 4: Streamlit UI with streaming

import streamlit as st

st.title("📚 Chat with Your Docs")

uploaded = st.file_uploader("Upload PDF", type="pdf")

if uploaded and "chain" not in st.session_state:
    with st.spinner("Processing document..."):
        chunks = load_and_split(uploaded)
        vs = create_vectorstore(chunks)
        st.session_state.chain = create_chain(vs)
        st.session_state.messages = []
    st.success(f"Processed {len(chunks)} chunks")

if "messages" not in st.session_state:
    st.session_state.messages = []

for msg in st.session_state.messages:
    with st.chat_message(msg["role"]):
        st.write(msg["content"])
        if msg.get("sources"):
            with st.expander("Sources"):
                for s in msg["sources"]:
                    st.write(s)

if prompt := st.chat_input("Ask a question"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.write(prompt)

    with st.chat_message("assistant"):
        # Stream the response
        response = st.session_state.chain.invoke({"input": prompt})
        answer = response["answer"]
        sources = [d.page_content[:200] for d in response["context"]]

        st.write(answer)
        if sources:
            with st.expander("Sources"):
                for s in sources:
                    st.write(s)

    st.session_state.messages.append({
        "role": "assistant", "content": answer, "sources": sources
    })

Step 5: Streaming responses

For token-by-token streaming, use LangChain’s stream:

def stream_answer(chain, question):
    for chunk in chain.stream({"input": question}):
        if "answer" in chunk:
            yield chunk["answer"]

Then in Streamlit:

with st.chat_message("assistant"):
    response = st.write_stream(stream_answer(chain, prompt))

Step 6: Multiple documents

uploaded_files = st.file_uploader("Upload PDFs", type="pdf", accept_multiple_files=True)

if uploaded_files:
    all_chunks = []
    for f in uploaded_files:
        all_chunks.extend(load_and_split(f))
    vs = create_vectorstore(all_chunks)
    st.session_state.chain = create_chain(vs)

Step 7: Conversation memory

from langchain.memory import ConversationBufferMemory

memory = ConversationBufferMemory(
    memory_key="chat_history",
    return_messages=True,
    output_key="answer",
)

# Add memory to the chain
chain = create_retrieval_chain(retriever, document_chain)
# Use memory in the prompt template

For production, use ConversationSummaryMemory to compress long histories.

Step 8: Local/private version (Ollama)

Swap OpenAI for local models:

from langchain_community.embeddings import OllamaEmbeddings
from langchain_community.llms import Ollama

embeddings = OllamaEmbeddings(model="nomic-embed-text")
llm = Ollama(model="llama3.2")

100% local, no API keys. Quality depends on the model.

Common mistakes

  • Chunk size too big/small — 1000 chars with 200 overlap is a good default. Tune per document type.
  • Not showing sources — users need to verify answers. Always show retrieved chunks.
  • No memory management — long conversations overflow context. Trim or summarize.
  • Ignoring retrieval quality — if retrieval is bad, the LLM can’t answer well. Test retrieval separately.
  • Re-embedding on every query — embed once, store in Chroma, reuse. Don’t re-embed.

Key Concepts

  • RAG = Retrieve relevant chunks → Augment prompt → Generate answer
  • LangChain = framework for LLM apps (loaders, splitters, chains)
  • Retriever = finds relevant chunks from vector store
  • Stuff documents chain = puts retrieved docs into the prompt
  • Streaming = token-by-token output for responsive UI

What to Try Next

FAQ

LangChain vs building RAG from scratch?

LangChain = faster to build, good defaults, but abstracts away details. From scratch = full control, fewer dependencies, easier to debug. Start with LangChain; go custom when you need control.

LangChain vs LlamaIndex?

LlamaIndex is RAG-focused (better retrieval, more data connectors). LangChain is more general (agents, tools, chains). For pure RAG, LlamaIndex is often better. For agents + RAG, LangChain.

Is LangChain production-ready?

Yes, but it changes fast (breaking changes between versions). Pin your version. For production, consider LangChain’s langserve or building a custom pipeline. The concepts (load, split, embed, retrieve, generate) are stable even if the API changes.