How to Build an HR Policy Chatbot with RAG
For Internal tools developers at mid-size companies · Based on Sujan Anand RAG Application Build Framework
// TL;DR
This guide shows internal tools developers how to build an HR policy chatbot using the RAG Application Build Framework. Run the Offline Pipeline once on your HR policy PDF — extract, chunk, embed with text-embedding-3-small, and store in ChromaDB with page metadata. Then deploy the FastAPI backend and Streamlit frontend so employees can ask questions like 'How many sick days do I get?' and receive grounded answers that cite the exact policy page. The context-only prompt ensures the bot says 'I do not know' rather than inventing policies it can't find.
Why build an HR chatbot with RAG instead of a generic LLM?
A generic LLM will confidently invent HR policies it was never trained on — a compliance nightmare. The RAG Application Build Framework grounds every answer in your actual policy PDF. When an employee asks 'How many sick days do I get?', the system retrieves the exact chunk from your document, injects it into a context-only prompt, and generates an answer that cites the page. If the policy isn't in the document, the bot says 'I do not know' instead of guessing. This is the difference between a trustworthy internal tool and a liability.
How do you ingest the HR policy PDF?
Run the Offline Pipeline once. Extract text page by page with PyMuPDF, skipping blank pages and calling doc.close() to avoid memory leaks. Chunk the text into 500-character pieces with 50-character overlap so no sentence gets split at a boundary — critical for policies where a single clause spans a page break. Embed the chunks in batches of 100 with text-embedding-3-small, then store them in ChromaDB with metadata including the source filename and page number. That metadata is what powers the page citations employees will see.
How do employees ask questions and get grounded answers?
The Online Pipeline runs on every question. Embed the employee's question with the same model used for the chunks, then call collection.query with n_results=3 to retrieve the three most semantically relevant chunks. Because this is semantic search, an employee asking 'how long is parental leave?' will match a policy section titled 'Family and Caregiving Time Off' even without shared keywords. Join the retrieved chunks into a context block and inject them into the prompt: 'Answer the question using only the context below. Always mention the page the answer came from. If the answer is not in the context, say I do not know.' Then call gpt-4o-mini and return the answer plus sources.
How do you make the chatbot trustworthy for HR?
Trust comes from explainability. In the Streamlit frontend, display each answer with an st.expander showing the retrieved chunk text, the page number, and the similarity score. HR reviewers can then verify that every answer traces back to a real clause in the official document. Store chat history in st.session_state['messages'] so conversations persist across reruns. Deploy the FastAPI backend first with uvicorn, then launch the Streamlit app — employees upload the policy once via the sidebar, click Ingest PDF, and start asking questions.
What edge cases should you plan for?
Policies change. When you upload a new version, delete the existing ChromaDB collection first to avoid duplicate chunks and conflicting answers. Test deliberately with out-of-scope questions like 'What's the CEO's salary?' to confirm the bot returns 'I do not know' rather than hallucinating. Tune chunk size down toward 300 tokens if your policy document is dense and clause-heavy, which improves retrieval precision for specific entitlement questions.
Next step
Start by validating embeddings on a few representative policy sentences to confirm the model distinguishes related from unrelated clauses. Then run the full Offline Pipeline on your HR PDF, wrap it in the FastAPI backend, and ship the Streamlit chat UI to a small pilot group before rolling out company-wide.
// FREQUENTLY ASKED QUESTIONS
How do I make sure the HR bot never invents a policy?
Use the context-only prompt instruction: 'Answer the question using only the context below. If the answer is not in the context, say I do not know.' This forces the LLM to answer only from retrieved chunks. Test with deliberately out-of-scope questions to confirm it returns 'I do not know' rather than a fabricated policy.
How do employees know which policy page an answer came from?
Store page number metadata with every chunk during ingestion, instruct the LLM to mention the page, and display the retrieved chunks, page numbers, and similarity scores in a Streamlit st.expander. Employees and HR reviewers can then verify each answer traces back to a real clause in the official document.
What happens when we update the HR policy document?
Delete the existing ChromaDB collection first, then re-ingest the new PDF to avoid duplicate chunks and conflicting answers. Because RAG retrieves from stored documents rather than model weights, your bot reflects the updated policy immediately after re-ingestion — no fine-tuning or retraining required.