Self-Host Dify: Internal RAG Chatbot for Your Docs

By the end of this playbook you will have:

1. VPS — 4 GB RAM minimum (8 GB comfortable); Docker + Compose v2.24+.
2. Disk for models/embeddings/files — start with 40+ GB free.
3. Domain optional for day one (dify.yourdomain.com); required before you share with the company.
4. LLM provider key (OpenAI, Azure, Anthropic, local OpenAI-compatible, etc.) that Dify’s model providers support in your version.
5. A folder of internal docs you are allowed to index (handbook PDF, SOPs, FAQ Markdown). No customer PII in v1 if possible.
6. Password manager for .env / SECRET_KEY / DB passwords.

git clone https://github.com/langgenius/dify.git
cd dify/docker
cp .env.example .env

Edit .env essentials (names may evolve — check .env.example comments):

• Public URLs when you have a domain: CONSOLE_WEB_URL, CONSOLE_API_URL, APP_WEB_URL, SERVICE_API_URL, FILES_URL → https://dify.yourdomain.com variants as documented
• VECTOR_STORE — default Weaviate is fine for starters; switch only if you already run Milvus/OpenSearch (see envs/vectorstores/)
• Generate/set strong DB and Redis passwords if placeholders exist
• Leave SECRET_KEY empty to let Dify generate a persistent key, or set your own and vault it

Do not invent extra vendor features — stick to variables present in your checked-out .env.example.

docker compose up -d
docker compose ps

Wait until API/web/db/redis/worker/vector containers are healthy (first pull is large).

Open the console (default often http://SERVER_IP on port 80 via the bundled nginx — confirm published ports with docker compose ps). Create the first admin account.

1. In Dify console → Settings → Model Providers.
2. Add your provider + API key.
3. Set a default chat model and an embedding model suitable for RAG.
4. Run a tiny playground chat to prove the key works.

1. Knowledge → Create knowledge → name it Company Ops Docs.
2. Upload 3–10 clean documents (employee handbook excerpt, refund SOP, IT access FAQ). Prefer Markdown/PDF text you own.
3. Choose chunking defaults; start with Dify’s recommended settings; re-index after changes.
4. Wait until indexing shows complete; open a chunk preview to ensure text extracted (scanned PDFs without OCR will fail silently).

1. 1

   Create Chatbot

   Studio → Create app → Chatbot (or Agent if you need tools later — start with Chatbot + Knowledge).
   Attach knowledge Company Ops Docs.
   System prompt (ops voice):

2. 2

   Test and Share

   Publish / get the web app URL or embed.
   Test three questions: one answered in docs, one partially answered, one unknown — confirm the unknown path does not hallucinate.

You are the internal ops assistant for COMPANY.
Answer ONLY using the knowledge base.
If the docs do not contain the answer, say you do not know and tell the user which team to ask.
Always cite which document the answer came from when possible.
Never invent policy, prices, or legal commitments.

1. Point DNS to the VPS; set console/app URLs to HTTPS in .env; restart.
2. Use Dify’s certbot compose path (docker/certbot/README.md) or put Cloudflare/Traefik in front.
3. Restrict who can create apps; disable public registration if exposed.
4. Prefer company SSO later; day one: strong admin password + few builder seats.

• [ ] docker compose ps healthy
• [ ] Admin account vaulted
• [ ] Model + embeddings work
• [ ] Knowledge indexed; sample chunks readable
• [ ] Chatbot refuses unknowns cleanly
• [ ] HTTPS + restricted audience
• [ ] Volume + .env backup exists
• [ ] Owner for doc freshness named

• Tiny VPS OOM during embedding — size up or index fewer docs first.
• URL mismatch — browser on IP while .env says a domain → blank API errors.
• Scanned PDFs — no text → empty RAG; OCR offline first.
• Stale knowledge — bot cites last quarter’s policy; schedule re-uploads.
• Public app link — treat like an internal wiki; put Access in front.

Dify vs plain chatbot playbook?
This stand-up owns the RAG platform. Pair with AI Customer Support Chatbot patterns for escalation design.

Can agents call n8n?
Yes in later iterations via tools/webhooks — keep v1 read-only Q&A.

Solutions · Architecture · Contact.