Give Your Private AI a Discord Bot Your Family Will Actually Use

You built a private AI assistant on your own hardware, you gave it your house manual, and it answers beautifully. So why is nobody at home using it?

If that sounds familiar, the problem is almost never the model. It is the doorway. Asking a family member to remember a URL, open a browser tab, sign in, and pick the right model from a dropdown is asking them to change their habits, and habits rarely lose. Instead of making everyone come to the assistant, put the assistant where they already are: in a chat app that is already open on their phone.

In my house that app is Discord, and this guide walks you through the bridge I built for it. By the end you will have a small Discord bot that takes !ask messages from a family server, forwards them to a self-hosted Open WebUI instance backed by Ollama, and posts the grounded answer back into the channel.

Moreover, the detour is worth taking on its own merits. Along the way you will work with REST API integration, Docker networking, GPU memory budgeting, and retrieval-augmented generation, which is a genuinely transferable set of skills for a project you can finish over a weekend.

One assumption before we start: this guide picks up where the private AI leaves off. It is part two of this series, so if you do not have an assistant running yet, part one on deploying your private AI with Ollama builds that half of the stack, and everything below is the layer that finally makes it useful to everybody else in the house.

The Stack: A Deliberately Boring Bridge

#

The design principle behind the whole thing is that the bot should be boring. It does no AI work at all: it does not embed documents, it does not manage prompts, and it does not talk to Ollama. It only reads Discord messages and makes HTTP calls. That keeps the interesting parts (the system prompt, the knowledge base, the model choice) in a single place, where you can change them without touching one line of Python.

Family Discord server                    the interface everyone already has
        │  !ask Where is the water shutoff valve?
        ▼
Discord bot (Python, Docker)             a thin bridge, no AI logic
        │  POST /api/chat/completions    Authorization: Bearer <key>
        ▼
Open WebUI (Docker)                      system prompt + house manual knowledge base
        │  OLLAMA_BASE_URL
        ▼
Ollama (native service, GPU)             the model runtime

Everything except Discord itself runs in WSL2 on my desktop PC, which is where the GPU lives: an NVIDIA GeForce RTX 3070 Ti with 8 GB of VRAM, the same card from the PC I built myself. That 8 GB number will come back later, because it quietly decides which model you can run. Open WebUI and the bot are deliberately co-located in the same WSL instance with network_mode: host, so they reach each other and Ollama over localhost with no cross-host networking to debug. My home server was the obvious candidate for hosting the bot, although it has about 1.8 GB of RAM and is already busy, so keeping the three pieces together on the machine with the GPU turned out to be both simpler and faster.

One property is worth calling out, because it shapes the whole security story: the bot is outbound only. It dials out to Discord and out to Open WebUI, and nothing on the internet ever connects into my house to reach it. That was the deciding factor when I compared Discord against WhatsApp, whose official Cloud API expects an inbound webhook, while the unofficial libraries carry both a ban risk and a second runtime to maintain.

Step 1: Create the Discord Bot

#

Head to the Discord Developer Portal and create a new application, then open the Bot tab.

Three things matter here:

1. Copy the token and keep it somewhere safe. It is a password for your bot, and it goes into a .env file that never gets committed.
2. Enable the Message Content Intent under Privileged Gateway Intents. Without it your bot connects successfully, sits in the channel, and silently ignores every message, because Discord will not send it the text. It is the single easiest way to lose an afternoon on this project, so do it now.
3. Invite the bot to your server with Send Messages, Read Message History, and Embed Links. Nothing more is needed, and giving a home automation bot fewer permissions is always the right instinct.

Step 2: Put the Knowledge in Open WebUI, Not in the Bot

#

This is the decision that keeps the project small, so it is worth doing before writing any code.

Inside Open WebUI, go to Workspace → Models and create a custom model. Mine is called Family1, and it carries two things: a system prompt written in Spanish that tells the model it is a household assistant, and a knowledge collection with the house manual uploaded to it. That manual is a plain Markdown document describing the Wi-Fi setup, the smart home devices, the breaker panel, the appliances, and all the small pieces of knowledge that normally live in one person’s head.

Open WebUI handles the retrieval on its own. When a question arrives, it pulls the relevant chunks from the manual and hands them to the model as context, which is what makes the answers specific to your house instead of generically plausible. The important consequence is architectural: the prompt and the knowledge live in Open WebUI’s data volume, not in Ollama and not in the bot. That means you can rewrite the assistant’s personality or upload a new revision of the manual without rebuilding or restarting anything.

That same fact is also a warning. Everything that makes the assistant yours sits in one Docker volume, so back it up before any teardown:

docker run --rm -v open-webui:/data -v $PWD:/backup alpine \
  tar czf /backup/owui-data.tgz -C /data .

Finally, generate the credential the bot will use: Settings → Account → API Keys → Generate new key. Copy it, because you cannot view it again later.

Step 3: The Bridge Between Discord and the Open WebUI API

#

Now the code. The whole bot is one Python file with two commands, and that is not an accident. Every feature I was tempted to add turned out to belong upstream in Open WebUI instead.

The dependencies are minimal:

discord.py>=2.3,<3
requests>=2.31,<3
python-dotenv>=1.0,<2

The core of it is a single function that posts a question and pulls the answer out of the response:

# OpenWebUI uses the OpenAI-compatible chat completions endpoint.
# Previously this was /api/chat (wrong) — the correct path is /api/chat/completions.
ASK_ENDPOINT = f"{OPENWEBUI_URL}/api/chat/completions"


def ask_openwebui(question: str) -> str:
    """Send a question to OpenWebUI and return the model answer."""

    headers = {"Content-Type": "application/json"}
    if OPENWEBUI_API_KEY:
        headers["Authorization"] = f"Bearer {OPENWEBUI_API_KEY}"

    # No system message here on purpose: the OpenWebUI model "Family1" already
    # carries its own (Spanish) system prompt + the "Manual Casa" knowledge base.
    # Sending a second system message here competes with / overrides that prompt.
    payload = {
        "model": OPENWEBUI_MODEL,
        "messages": [
            {"role": "user", "content": question},
        ],
        "stream": False,
    }

    log.info("POST %s  model=%s", ASK_ENDPOINT, OPENWEBUI_MODEL)
    response = requests.post(ASK_ENDPOINT, json=payload, headers=headers, timeout=180)

    if not response.ok:
        log.error(
            "OpenWebUI returned HTTP %d: %s",
            response.status_code,
            response.text[:400],
        )
    response.raise_for_status()

    data = response.json()

    # Try a few common response shapes so the script is easier to adapt.
    # OpenWebUI's /api/chat/completions returns the standard OpenAI shape:
    # {"choices": [{"message": {"content": "..."}}]}
    if isinstance(data, dict):
        if "choices" in data and data["choices"]:
            return data["choices"][0]["message"]["content"].strip()
        if "message" in data and isinstance(data["message"], dict):
            content = data["message"].get("content")
            if content:
                return str(content).strip()
        if "content" in data and isinstance(data["content"], str):
            return data["content"].strip()

    return str(data)

A few details in there are load-bearing, and each one came from getting it wrong first.

Use the OpenAI-compatible path. Open WebUI exposes /api/chat/completions, not /api/chat. My first version used the latter and got nothing but errors that looked like an authentication problem.

Keep OPENWEBUI_URL a bare base URL. The script appends the path itself, so putting a path in the variable quietly doubles it and produces a confusing 404.

Do not send a system message. It feels natural to define the assistant’s persona in code, but doing so competes with the prompt already attached to the custom model, and the result is an assistant with two contradictory sets of instructions.

Parse defensively. Open WebUI returns the standard OpenAI shape, but checking for a couple of alternative shapes costs a handful of lines and makes the script easy to point at a different backend. Logging the status code and the first few hundred characters of a failed body costs about as little, and it is the difference between debugging from evidence and debugging from a shrug.

The Discord side is equally small. !ask runs inside a typing indicator, so the family can see the bot is working rather than assuming it is broken, every failure path ends in a sentence a human can read, and the answer is chunked before sending because Discord rejects messages over 2000 characters:

@bot.command(name="ask")
async def ask(ctx: commands.Context, *, question: str) -> None:
    """Ask the family assistant a question.

    Usage:
        !ask How do I turn on movie mode?
    """

    async with ctx.typing():
        try:
            answer = ask_openwebui(question)
        except requests.RequestException as exc:
            status = getattr(getattr(exc, "response", None), "status_code", None)
            detail = f" (HTTP {status})" if status else ""
            await ctx.send(f"Sorry, I could not reach OpenWebUI{detail}: {exc}")
            return
        except Exception as exc:  # noqa: BLE001 - show a friendly error message
            await ctx.send(f"Something went wrong: {exc}")
            return

    if not answer:
        await ctx.send("I did not get an answer back.")
        return

    # Discord message limit is 2000 characters.
    if len(answer) <= 2000:
        await ctx.send(answer)
        return

    # If the answer is long, split it into chunks.
    chunk_size = 1900
    for start in range(0, len(answer), chunk_size):
        await ctx.send(answer[start : start + chunk_size])

A !ping command that replies pong rounds it out. It sounds trivial, but it answers the most common household question (“is it down, or is it just slow?”) without anyone needing to read a log.

Step 4: Containerize and Run

#

The image is about as simple as a Python image gets, with the dependency layer cached separately from the source so code edits rebuild in seconds:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY family_discord_openwebui_bot.py .
CMD ["python3", "family_discord_openwebui_bot.py"]

And the Compose file:

services:
  bot:
    build: .
    container_name: family-assistant-bot
    restart: unless-stopped
    env_file: .env
    network_mode: host

network_mode: host is what lets the container reach Open WebUI at plain localhost:8080, since both live in the same WSL instance. restart: unless-stopped means the bot comes back on its own after a reboot, which matters for something the household is supposed to be able to rely on.

Your .env holds four values:

DISCORD_TOKEN=your_discord_bot_token
OPENWEBUI_URL=http://localhost:8080
OPENWEBUI_MODEL=ollama-family1:latest
OPENWEBUI_API_KEY=sk-...

OPENWEBUI_MODEL deserves a second look, because it is the subtlest trap in the whole project. It must be the id of the custom model you created in Step 2, the one with the house manual attached. If you point it at the bare engine instead, everything appears to work: the bot connects, questions get answers, the answers are fluent. They are also completely ungrounded, because you have bypassed both the system prompt and the retrieval. Confirm the exact id under Workspace → Models rather than guessing, and do not copy it from my repository’s .env.example, which still shows the engine id there and is exactly the mistake this paragraph is warning you about.

Then start it:

docker compose up -d
docker compose logs -f bot

The logs should show Logged in as ... and Bot is ready. In Discord, !ping should return pong, and !ask should return something only your own manual could have told it.

The Hard Parts: Four Obstacles Worth Knowing About

#

The bridge itself took an evening. Everything underneath it is where the real engineering went, and the four lessons below generalize far beyond this project, so they are the part I would read first if I were you.

The GPU that was not being used

#

For a while the assistant answered correctly but painfully slowly, and the logs explained why: Ollama kept falling back to library=cpu, reporting failure during GPU discovery ... failed to finish discovery before timeout after about thirty seconds, for both CUDA 12 and CUDA 13. Meanwhile nvidia-smi inside WSL happily listed the card. CUDA looked healthy and the GPU looked present, yet inference ran on the CPU.

Tracing the process revealed the actual mechanism. Ollama discovers GPUs by spawning a runner subprocess that listens on 127.0.0.1 on a random port, then connecting to it over loopback to ask what hardware exists. My WSL was configured with networkingMode=mirrored, and under mirrored mode those local process-to-process connections route through the Windows network stack and hang. The parent’s connect() sat unresolved until the timeout expired, the runner was killed, discovery “failed”, and Ollama concluded there was no GPU.

Two quick experiments confirmed it. A five-line Python script that listened and connected on 127.0.0.1 hung in exactly the same way, and a small PyTorch matrix multiply finished in 0.4 seconds on the GPU, which placed the fault in loopback rather than in CUDA or Ollama.

The fix was one line in C:\Users\fdeze\.wslconfig:

[wsl2]
networkingMode=NAT
localhostForwarding=true

After wsl --shutdown, Ollama loaded the model at 100% GPU. As a bonus, NAT also restored my university VPN, which mirrored mode had quietly broken. The lesson I keep coming back to: when a symptom points at the exotic layer (drivers, CUDA, the GPU), verify the boring layer first. It was networking.

Choosing a model that actually fits

#

My first engine was qwen3.5-tuned, a Qwen3.5 9.7B quantized to Q4_K_M with an 8192 token context. On paper it fit, and ollama ps cheerfully agreed: PROCESSOR read 100% GPU. The arithmetic is what tells the real story. That engine occupies 6.6 GB, the Windows desktop is already holding about 1.3 GB of the card before Ollama asks for anything, and 6.6 plus 1.3 lands at roughly 7.9 GB of the RTX 3070 Ti’s 8.2 GB. That is around 250 MB of headroom on a card that is 97 percent full.

Living at 97 percent is not a stable place to be. At rest the model was perfectly reproducible, generating at about 70 tokens per second across repeated runs. But while I was benchmarking, the very same model on the very same prompt intermittently collapsed to roughly 20 tokens per second, with prompt processing dropping from about 800 tokens per second to about 133, whenever the Windows desktop reached for VRAM and the last 250 MB were simply not there. That is the spill: the KV cache and compute buffers move to system memory and the work quietly stops being GPU work. An occasional, unpredictable slowdown is a worse property for a family service than a permanent one, because nobody can tell whether it is broken or just having a bad minute. Being a reasoning model made it worse still, because the long internal reasoning trace was precisely the part running at CPU speed.

The answer was not a bigger card, it was right-sizing. I built a smaller engine from a 4B base with a 4096 token context:

FROM qwen3.5:4b
PARAMETER num_ctx 4096
PARAMETER num_gpu 99
PARAMETER temperature 0.7
PARAMETER top_k 20
PARAMETER top_p 0.9
PARAMETER presence_penalty 1.5
PARAMETER repeat_penalty 1.1

ollama create qwen3.5-4b-tuned -f qwen-family-4b.Modelfile
ollama ps   # the acceptance test: PROCESSOR must read 100% GPU

The 4B engine occupies 5.5 GB, which leaves 2.7 GB of the card unclaimed by the model and still about 1.4 GB genuinely free once the desktop takes its share. That margin is the whole point, and the numbers followed it: the 4B holds around 103 tokens per second, against the 9.7B’s 70 at its best, and it holds that rate consistently rather than sometimes. Right-sizing did not cost me speed, it bought speed.

So the real acceptance test is not PROCESSOR reading 100% GPU, since the oversized engine passed that too. It is 100% GPU with roughly a gigabyte of VRAM still free, and that free gigabyte is the difference between an assistant the family trusts and one they give up on. Although dropping from 9.7B to 4B sounds like a downgrade, the quality cost turned out to be small, precisely because the answers are grounded in the house manual rather than in the model’s own memorized knowledge. Retrieval let me spend my VRAM on speed instead of on trivia.

That argument does carry an assumption I had not tested, though: it only holds while the retrieval actually finds the right passage. I come back to that further down, because when I finally measured it, it turned out to be the weakest link in the whole stack.

Turning off the thinking

#

The last obstacle was the strangest. My engine reasons by default, emitting hundreds of internal tokens even for a greeting, and combined with a large system prompt and retrieved manual chunks inside a 4096 token budget, that reasoning trace ate the answer. I did not want to leave that as an impression, so I reproduced it: the same question, the same seed, the context deliberately filled to the model’s full 4096 tokens to match the 1,547 to 2,284 tokens a real question carries after retrieval, run once with thinking off and twice with it on.

That table is the whole failure in one place. With thinking on, the reasoning trace consumed the output budget before the answer ever started: the better of the two runs still took 51 seconds to deliver 464 characters, and the other spent 92.9 seconds emitting 29,254 characters of reasoning and then returned nothing at all. A blank message after a minute and a half is not a slow assistant, it is a broken one, and anyone in the house would reasonably conclude exactly that.

One nuance is worth stating, because it explains why this is specifically a retrieval problem: thinking is only catastrophic when the context is full. On a short unconstrained prompt the same engine merely slows from roughly 10 to 14 seconds up to 27 to 30. It is the combination of a large system prompt, retrieved chunks, and a 4096 token ceiling that turns a slowdown into an empty reply. That is why turning it off is mandatory on a RAG path rather than a nice optimization.

The widely suggested fix, putting /no_think in the prompt, did nothing on this build: I measured 479 reasoning tokens with the flag present. What worked was setting think: false as a parameter on the custom model in Open WebUI, under Workspace → Models → Params. It has to live on the model definition, because Open WebUI’s chat completions endpoint does not forward a request-level think field from the bot.

With that in place, answers land in about four seconds. I went back and measured it properly rather than trusting the impression: nine questions sent end to end through Open WebUI against the Family1 model with the house manual attached, each one carrying between 1,547 and 2,284 prompt tokens after retrieval and producing between 75 and 601 tokens of answer. The fastest came back in 2.3 seconds, the typical one in three to six, and the slowest, which was the first request of the run, in 8.4 seconds. For a household question asked from a phone, that is indistinguishable from instant.

Cold starts used to be the frightening part. When the model still had to load and run its first retrieval, the first question of the day took 100 to 134 seconds, which is exactly why the bot’s HTTP timeout is 180 seconds rather than the 90 I started with: I was timing out on requests that were going to succeed. Those numbers no longer reproduce at all. I unloaded the Ollama engine and restarted the open-webui container to clear its caches, then timed four questions in a row, and the genuinely cold one came back in 9.8 seconds, followed by 6.2, 6.2 and 5.4 warm.

I have kept the generous timeout anyway, because a timeout you never reach costs nothing, while one that is slightly too short costs you the exact request you most wanted to work. Every measurement in this post, along with the benchmark scripts that produce them, lives in BENCHMARKS.md in the bot’s repository, linked at the end of this post, so you can rerun them against your own hardware instead of taking my word for any of it.

Surviving a reboot

#

NAT gave WSL a private IP that changes on every boot, which would break LAN access to Open WebUI. A netsh portproxy rule on Windows forwards the host’s ports 8080 and 11434 to the current WSL address, and because that address moves, a small PowerShell script refreshes the rules and opens the matching firewall entries, run by a scheduled task at logon. The Windows host IP never changes, so my reverse proxy configuration and public hostname never need touching.

The Weakest Link: Measuring Whether the Retrieval Works

#

Everything above rests on a claim I had never actually tested: that grounding the answers in the house manual is what lets a 4B model do this job. If that claim is true, retrieval is the most important component in the stack, which means it deserves a measurement of its own. So I finally gave it one, and the honest result is that it is also the weakest part of what I built.

The trick is to score retrieval with the model out of the room. For each of sixteen questions I asked Open WebUI for the top k chunks and checked whether the correct answer appeared verbatim in what came back. No generation, no judging of prose, just a yes or a no about whether the fact ever reached the context window at all. That separation matters, because a wrong answer from a RAG system has two completely different causes, and you cannot fix the one you have not identified.

Production runs with TOP_K = 3, so the number that describes my house is 11 out of 16. Roughly a third of the time the answer never reaches the model at all. When that happens the assistant is not wrong, it is uninformed, and from the outside those two failures look exactly alike.

What surprised me was how little the obvious knobs moved. Raising k bought at most one question, and cleaning up the manual bought at most one. The reason is a remarkably clean split in the rankings: every question the retriever gets right, it gets right within the top four chunks, and every question it gets wrong sits at rank 10 or worse, at 15, 17, 19 and 24 on the production manual. There is nothing in between. No realistic k rescues those failures without first dragging ten chunks of noise into a 4096 token budget, which is the same context pressure that made the reasoning trace so destructive earlier.

The failures also have a shape, and that is what points at the fix. All four of the persistent ones are proper nouns living inside markdown tables: Wi-Fi network lookups, a hardware model number, that kind of question. This is precisely the query that keyword search handles well and dense vector embeddings handle badly, because an exact token match earns almost no extra credit in a space built to represent meaning. The lever is therefore not more chunks, it is better ordering: a cross-encoder reranker over a wider candidate list. Turning on hybrid search by itself is a no-op on my instance, since with no reranking model configured the merged candidates simply get re-scored by the same similarity metric that ranked them in the first place.

One more finding is worth carrying to your own knowledge base, because it costs nothing. My production manual is 836 KB, of which only about 17 KB is text. The rest is screenshots embedded as base64, which is why 698 of its 744 chunks are image noise instead of prose. Stripping them improved the rankings broadly even where the final verdict did not change. A knowledge base is not a folder you dump documents into, it is a document you maintain.

So the reflection I keep arriving at applies here too: retrieval is what let a small model do a big model’s job, and retrieval is also where the most headroom is left. That is worth knowing before you assume that attaching a document is the end of the work. The numbers are in section 6 of that same BENCHMARKS.md, and the scoring script is bench/bench_rag.py, so you can point it at your own knowledge base and find out what your assistant can actually see.

What You Get Out of This

#

The everyday payoff is unglamorous, and that is exactly the point. Someone types !ask in a Discord channel they already had open, and three to six seconds later they know where the water shutoff valve is, how to get the projector back on the right input, or what the Wi-Fi password is. In my house nobody had to be taught a new tool, and nobody had to wait for me to answer, which is the closest thing to success a household project can have.

The engineering payoff is a stack with clean seams. The bot is stateless on purpose: each !ask sends only that one question, so follow-ups like “and how do I change it?” do not work. That is a deliberate trade rather than an oversight. Statelessness keeps the entire 4096 token context available for the system prompt and the retrieved manual chunks, and a household FAQ is overwhelmingly made of standalone questions. I will add per-channel history the day someone actually asks for it, and not before.

Further, the skills you pick up here transfer everywhere: REST API integration and defensive response parsing, Docker networking and container lifecycle, GPU memory budgeting, and the kind of methodical debugging that separates a symptom from a cause. Every obstacle along the way (the intent I forgot to enable, the endpoint that was wrong, the loopback that hung, the model that did not fit) turned into a piece of understanding I still carry forward. Each setback is simply another opportunity to learn from past mistakes and keep improving, and in a home lab those lessons compound faster than almost anywhere else.

What’s Next

#

A few directions I think are worth exploring from here:

• A reranker in front of the retrieval. The failures I measured are ranking failures rather than budget failures, so the change that matches them is a cross-encoder rerank over a wider candidate list, not a larger k. It is the single item on this list I expect to move the assistant most.
• Conversational follow-ups. A short per-channel history in the messages array would enable natural back and forth, at the cost of context budget. Worth doing only alongside a larger context window.
• Slash commands. Discord’s native / commands with autocomplete would be friendlier than a ! prefix for family members who never learned the prefix convention.
• Write access, carefully. The assistant currently only answers. Letting it act on the smart home would be the natural next step, and it is the step that most needs guardrails: an explicit allowlist of actions, and confirmation before anything with physical consequences.
• Backups that actually run. The system prompt and the house manual live in a single Docker volume. A scheduled snapshot of it is the least glamorous and most valuable item on this list.

The full bot is on GitHub at EmilianFC20/family-assistant. If you build your own version, I would love to hear which questions your household ends up asking it most.