# Adding a low-cost chat assistant to my blog

- URL: https://baeke.info/2026/10/31/adding-a-low-cost-chat-assistant-to-my-blog/
- Author: Geert Baeke
- Published: October 31, 2026
- Topics: ai, azure, chatbots, foundry, static-web-apps

Building chatbots is a big part of my day job, so it was a bit odd that my own blog didn't have one. Now that baeke.info is a [static site on Azure](https://baeke.info/2026/10/28/from-wordpress-com-to-a-static-site-on-azure/), adding one was easy. There was one requirement though: it should cost next to nothing. A chat box on a public website is an open invitation to send it anything, and every message that reaches a large language model costs money.

So I built **Ask the blog**: a small chat that answers questions from the posts and links to them. It sits behind the orange button at the bottom right of every page:

![The baeke.info home page with the orange chat button at the bottom right](https://baeke.info/_astro/button.BGc0hDVf_ZXCrbM.webp)

Click it and the chat opens. Here it is with an answer, a follow-up, and a polite "no" to small talk:

![The Ask the blog chat with an answer and a source link, a follow-up question, and the fixed reply to "ok, thanks!"](https://baeke.info/_astro/chat.CuKegCgd_Z1K2AhS.webp)

In this post I'll go through the design decisions that keep it cheap. The short version: **run the cheapest checks first**, and only let a question reach the expensive model when it is worth answering. The repo is private for now, so this post stays at the level of the design.

---

## Cheapest checks first

Every question goes through a series of checks. Each check can stop it with a fixed reply, and the early ones cost nothing at all:

![Cheapest checks first: free checks, the Decision-1 gate and the budget, then the answer from gpt-6-luna](https://baeke.info/_astro/gate.CPoRUdKN_Zlj0NB.webp)

The whole thing runs as managed Functions next to the site, on the Free plan of Azure Static Web Apps. No extra servers, no database, and a typical answer costs about $0.0003.

## Limits before anything else

Before a question gets anywhere near a model, a few free checks run:

- **A switch.** An app setting turns the chat on or off within a minute, without a deploy. When it's off, the button disappears.
- **A bot check.** Cloudflare Turnstile runs once per visit, which keeps most scripts out.
- **Rate limits per IP.** 15 questions an hour and 30 a day per visitor, and 500 a day for the whole site. The counters live in Azure Table storage, and the IP addresses are stored as salted hashes, not as plain addresses.
- **A daily budget.** $0.50 a day. Before each answer, the worst-case cost is reserved first and the real cost is booked afterwards, so a burst of parallel questions can't sneak past the budget.

⚠️ **Important:** rate limits per IP are a blunt tool. People behind the same company proxy share one bucket. For a personal blog that's fine.

## Keyword search, not vectors

The usual recipe for a chat over documents is embeddings plus a vector database. I didn't do that here. At build time, a script splits every post into chunks of about 300 words (around 1,350 chunks in total), and the function searches them with **BM25**, classic keyword ranking. The index ships with the function, so retrieval costs nothing per question and there is no search service to pay for.

BM25 works well when the question uses the words the posts use, like "Container Apps sandboxes" or "MCP". It's weaker on paraphrases. I asked "How is this blog hosted?" and it found nothing, although there is a whole post about that. The post talks about Static Web Apps and a static site, and never uses the word "hosted". 😊 For now that's an acceptable trade-off. If it misses too often, embeddings are a few cents to build.

## A decision model as the gatekeeper

This is the part I like most. Before the chat model runs, **Microsoft-Decision-1** looks at the question. It's a model in Microsoft Foundry that doesn't write text. You give it some state and a few typed questions, and it returns decisions: probabilities for yes/no questions, a choice from a fixed set, or a score on a scale. That makes it a good fit for gates and filters, and it's cheap: one check costs about $0.00004.

The gate asks five yes/no questions about the latest question, read together with the last three turns of the conversation:

| Question | Why |
| --- | --- |
| Is it on topic? | AI, agents, Azure, Kubernetes, containers, the blog and its author |
| Is it a follow-up? | "does it work with MCP?" only makes sense after the question before it |
| Is it mixed? | an on-topic question with an unrelated one tacked on |
| Is it abuse? | prompt injection, role-play, "ignore your instructions" |
| Are the search results relevant? | no relevant posts means no answer worth paying for |

A question passes when it's on topic or a follow-up, and not mixed or abusive. Everything else gets a fixed reply, and the chat model is never called.

The **follow-up** and **mixed** questions came from testing. At first the gate only saw the new question, so a perfectly reasonable follow-up was blocked as off topic. And this one went straight through:

> in france, are there datacenters and what is the capital of france

Half of it is about Azure regions, and that half lifted the whole question over the on-topic threshold. A separate "mixed" check catches that, and as a second line of defense the chat model is told to answer only the part of a question the blog covers.

⚠️ **Important:** the gate returns probabilities, not explanations, and the wording of the questions matters. I keep a small set of labelled test questions (on topic, follow-ups, off-topic follow-ups, mixed, abuse) and run them after every change to the wording or the thresholds.

## No chit-chat

"Hello", "ok" and "thanks!" are friendly, but answering them with a language model costs money and helps nobody. The gate treats small talk as off topic, and the reply says so:

> No small talk, I'm afraid: every message costs a little.

It costs one gate check, and that's it.

## The answer itself

Questions that pass go to **gpt-6-luna**, a small and cheap model, with a short leash:

- **No reasoning** and at most 450 output tokens. Answers are short on purpose.
- **Only the 4 best chunks** from the search, plus the last three turns.
- **Answer from the excerpts only.** It cites them as [1], [2], links the posts, and says so when the excerpts don't cover the question instead of filling the gap with general knowledge. You can see that in the screenshot: the blog doesn't say whether sandboxes work with MCP, so the assistant doesn't either.

To keep an eye on all this, there's a small usage page, only for me behind a sign-in. It shows the questions, what was answered or blocked and why, and the spend against the budget, per day, week or month. Here it is on launch day, when the only visitor was me, testing:

![The chat usage page for one day: 77 questions, 15 answered, 20 stopped by the gate, $0.0097 spent of the $0.50 budget, and a chart of questions per hour](https://baeke.info/_astro/usage.DwAEnOYY_ZypcQ7.webp)

Twenty of those questions never reached the chat model, and the whole day of testing cost less than a cent. 😎

## What you don't get

A few honest caveats:

- **It only knows the blog.** No web search, no general knowledge. That's the point, but it does mean "I couldn't find that" happens.
- **Keyword search misses paraphrases.** See the hosting example above.
- **The gate is not perfect.** Borderline questions can go either way. A question that is blocked wrongly costs a reader a retry, and one that slips through costs me a fraction of a cent.
- **Limits are shared.** A burst of visitors from one IP address, or a busy day for the whole site, ends with a friendly "you've reached the limit".

## Wrapping up

A chat on a public site doesn't have to be expensive. Put the free checks first (a switch, a bot check, rate limits, a budget), use plain keyword search instead of a vector database, and let a decision model decide which questions are worth an answer. Only what's left reaches the language model, with a short context and a short answer. On a normal day the whole thing costs a few cents.

The code lives in my private blog repo for now. Try it yourself with the button at the bottom right, and ask it something about the blog. 😎

If you spot any errors or have questions, feel free to reach out!