# Hybrid search without a vector database

- URL: https://baeke.info/2026/11/01/hybrid-search-without-a-vector-database/
- Author: Geert Baeke
- Published: November 1, 2026
- Topics: ai, azure, embeddings, rag, search, foundry

Last week I added a [low-cost chat assistant](https://baeke.info/2026/10/31/adding-a-low-cost-chat-assistant-to-my-blog/) to this blog. It found posts with BM25, plain keyword search, and I admitted it had a weak spot: ask "How is this blog hosted?" and it misses the post about hosting, because that post never uses the word "hosted". The usual fix is embeddings, and the usual advice is to store them in a vector database. I added the embeddings but skipped the database. All 1,362 vectors live in a 1.4 MB file that ships with the function, and the function compares every question with all of them.

BM25, embeddings, cosine similarity and reciprocal rank fusion are all you need for this, and none of them is hard. I timed a brute-force scan up to a million vectors and tested whether hybrid search finds better posts than BM25 on 45 questions. The code lives in my private blog repo for now, so the snippets are trimmed versions of the real thing.

The search path for one question:

![Hybrid search for one question: BM25 over the chunks and a scan over all vectors run side by side, reciprocal rank fusion merges the two lists, and the best 4 chunks go to the chat model](https://baeke.info/_astro/hybrid.eXeA-Skb_1VAPjR.webp)

---

## What is BM25?

BM25 is the classic way to rank documents for a keyword query. It's been around since the nineties and is still the default in Elasticsearch, Azure AI Search and most other search engines. For every word in the question, it looks at three things:

- How often the word appears in the chunk. More is better, with diminishing returns: the tenth "Kubernetes" adds less than the first.
- How rare the word is across all chunks. "Kopf" appears in one post, "Azure" in hundreds, so a match on "Kopf" counts for more.
- How long the chunk is. A match in a short chunk beats the same match in a long one.

The two classic knobs are **k1**, which controls how fast repeated words stop adding to the score, and **b**, which controls how much chunk length matters. The chat uses [MiniSearch](https://github.com/lucaong/minisearch), a small JavaScript library, with its defaults (k1 = 1.2, b = 0.7). I added two things on top: a match in the post title counts 3× and a match in a heading 2×, and the index drops common words like "how" or "what". Words of five letters or more also match with a typo.

BM25 needs no model and handles exact terms well. Ask about "k3sup" or "smolagents" and it finds the one post that mentions them. It matches words, though, and ignores what they mean. Ask "What is the smallest base image I can use for a compiled program?" and BM25 returns my posts about searching photos with GPT-4 Vision, because those mention "image" a lot. The post that answers the question talks about distroless and scratch containers for Go. 🤷

## What are embeddings?

An embedding model turns a piece of text into a list of numbers, a **vector**, in a way that captures meaning. Texts about similar things end up with similar vectors, even when they share no words. "Run containers without a daemon" and a post about Podman end up close together, and a question in Dutch ends up close to the English post that answers it.

I use **text-embedding-3-large** in Microsoft Foundry. It returns 3,072 numbers by default, but the text-embedding-3 models accept a `dimensions` parameter that returns a shorter vector. The models were trained so that the first numbers carry most of the meaning, and according to OpenAI's benchmarks even a shortened vector does slightly better than the 1,536 numbers of the older text-embedding-ada-002. I use **256**, which keeps the whole set small: 1,362 chunks × 256 numbers × 4 bytes is 1.4 MB.

A build script splits every post into chunks of about 300 words at its H2 and H3 headings. I embed each chunk together with its post title and heading, so a chunk that only says "Now deploy it" still carries its subject. Embedding the whole blog, about 330,000 tokens, cost less than five cents.

## Comparing vectors with cosine similarity

To find the chunks closest to a question, you embed the question with the same model and compare its vector with every chunk's vector. The usual measure is **cosine similarity**: the cosine of the angle between two vectors. Pointing the same way gives 1, unrelated gives something near 0.

![Cosine similarity in two dimensions: the question vector is at a small angle to a chunk about Podman (similarity 0.82) and at a wide angle to a chunk about Grafana dashboards (0.21)](https://baeke.info/_astro/cosine.Uwtt1bDB_ZNJYFk.webp)

The real vectors have 256 dimensions instead of two, and the math stays the same. The embedding service returns vectors with length 1, and for those the cosine similarity equals the **dot product**: multiply the numbers pair by pair and add them up. That loop is the whole "vector search":

```js
// The chunks closest to the query vector, best first
function nearest(query, k) {
  const { dims } = data.vectors;
  const scored = [];
  for (let i = 0, o = 0; i < data.chunks.length; i++, o += dims) {
    let s = 0;
    for (let d = 0; d < dims; d++) s += query[d] * vectors[o + d];
    scored.push([i, s]);
  }
  return scored.sort((a, b) => b[1] - a[1]).slice(0, k);
}
```

The function runs that loop over all 1,362 vectors for every question, without an index or approximate nearest neighbor search.

## Where do the vectors live?

In a file called `vectors.bin`: the raw bytes of one `Float32Array`, one row of 256 floats per chunk, in the same order as the chunks in `chunks.json`. Row 17 is the embedding of chunk 17, so the file needs no header or ids. The function reads it once when it starts:

```js
const buf = readFileSync(new URL('../../data/vectors.bin', import.meta.url));
vectors = new Float32Array(buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength));
```

The build writes that file, but the build never calls the embedding model. The embeddings live in a cache in git, `embeddings.jsonl`, with one line per chunk keyed by a hash of the text that was embedded. After writing a post I run `npm run embed`, which embeds only the chunks that are new or changed (a new post is a few dozen chunks, a fraction of a cent) and drops the ones no longer used. The build then turns the cache into `vectors.bin`. If I forget, the new chunks get a row of zeros and BM25 still finds them.

## How fast is a brute-force scan?

Scale is the usual argument for a vector database, so I measured it. The benchmark compares a query with every vector, for the blog's real vectors and for made-up collections up to a million chunks. It ran on my machine (a Ryzen 7 8745HS, Node 24, one thread):

![Time to compare one question with every vector, on a log scale: 0.33 ms for the blog's 1,362 chunks, 2.5 ms for 10,000, 20 ms for 100,000 and 199 ms for a million, against 134 ms for the embedding call itself](https://baeke.info/_astro/timings.DZCsAmJL_ZvAuk5.webp)

For the blog itself, the scan takes **0.33 ms**. The call that embeds the question takes **134 ms** (the median of 20 calls). The vector math is about 0.3% of the time hybrid search adds. Loading the 1.4 MB file at startup takes a fraction of a millisecond.

The full table, for 256 dimensions and for the full-size vectors:

| Chunks | Dimensions | Size | Scan |
| --- | --- | --- | --- |
| 1,362 (this blog) | 256 | 1.3 MB | 0.33 ms |
| 10,000 | 256 | 10 MB | 2.5 ms |
| 100,000 | 256 | 98 MB | 20 ms |
| 1,000,000 | 256 | 977 MB | 199 ms |
| 10,000 | 3,072 | 117 MB | 23 ms |
| 100,000 | 3,072 | 1.2 GB | 235 ms |

Scan time grows linearly with chunks × dimensions. The sort surprised me: at a million chunks, sorting all the scores (as `nearest()` above does) takes 938 ms, more than the 199 ms of math. Keeping only the best 50 while scanning fixes that, and at 1,362 chunks the difference is too small to matter. 😊

⚠️ **Important:** these numbers come from my own machine. Azure Functions on a Static Web App run on slower, shared CPUs, so expect a few times more. Even at 10× slower the blog's scan takes 3 ms.

## Hybrid search: combining both lists

Every question now gets two ranked lists, one from BM25 and one from the vector scan. BM25 nails exact names and rare terms, and embeddings catch paraphrases and other languages, so hybrid search merges the two.

Adding the scores doesn't work: a BM25 score of 14.2 and a cosine similarity of 0.43 sit on different scales. The common trick is **reciprocal rank fusion (RRF)**, which ignores the scores and uses the positions. Every chunk gets `1 / (k + rank)` from each list it appears in, and the totals decide the new order. A chunk near the top of both lists ends up first, and a chunk that only one list likes still gets a share. Azure AI Search does the same thing for its hybrid queries.

```js
// Reciprocal rank fusion: uses ranks only, so BM25 scores and cosine
// similarities never have to be put on one scale
function fuse(...lists) {
  const scores = new Map();
  for (const list of lists)
    list.forEach(([id], rank) => scores.set(id, (scores.get(id) ?? 0) + 1 / (RRF_K + rank)));
  return [...scores].sort((a, b) => b[1] - a[1]);
}
```

The constant **k** decides how much the top of each list counts. The usual value is 60, which comes from research on long result lists. A big k flattens the differences between ranks, so a chunk that is "pretty good" in both lists beats a chunk that is first in only one. That's a sensible default when you show a reader ten results. The chat only gives the model 4 chunks, though, and with k = 60 a question about "Decision-1", a model name the embeddings don't know, lost its perfect BM25 match. With **k = 5** the top of each list counts more, and that problem went away without hurting the other questions.

## The knobs

There are more settings than k, and most of them matter more:

| Setting | Value | What it does |
| --- | --- | --- |
| Chunk size | ~300 words, split at headings | Smaller chunks are more precise, bigger ones carry more context |
| Embedded text | title + heading + chunk | Gives every chunk its context |
| Dimensions | 256 of 3,072 | Size and speed against a little accuracy |
| BM25 k1 / b | 1.2 / 0.7 | Repeated words and chunk length |
| Field boosts | title 3×, heading 2× | A match in a title says more than one in the body |
| Vector candidates | 50 | How many nearest chunks take part in the fusion |
| RRF k | 5 | How much the top of each list counts |
| Per post | at most 2 chunks | One long post can't crowd out the rest |
| To the model | 4 chunks | Context size, and so cost |

Follow-up questions add one more. For "does it work with MCP?", the previous question is blended in with a lower weight (a quarter in the question's vector, a third for its words in BM25), so the follow-up keeps its subject while the new words decide which part of a post comes back.

## Did it help?

Speed tells you nothing about quality, so I wrote 45 questions and noted for each one which posts answer it:

- 20 paraphrases, written without the words the posts use, like "Can I run containers without a background daemon?" (Podman) or "Can I write my own Kubernetes controller in Python?" (Kopf).
- 15 keyword questions with exact names, like "k3sup", "kubelogin" and "Decision-1 gate".
- 10 questions in Dutch, because some readers ask in Dutch while the posts are in English.

A question counts as answered when one of its posts is among the 4 chunks the model gets:

| Questions | BM25 | Hybrid | Vectors only |
| --- | --- | --- | --- |
| Paraphrases (20) | 13 | 18 | 19 |
| Keywords (15) | 15 | 15 | 14 |
| Dutch (10) | 4 | 9 | 8 |
| All (45) | 32 | **42** | 41 |

Hybrid search answers 10 more questions than BM25, and it doesn't lose a single keyword question. A few that went from a miss to the top result:

- "What is the smallest base image I can use for a compiled program?" now finds the distroless post instead of the GPT-4 Vision ones.
- "How do I make a service on my laptop reachable from the internet?" finds the inlets post. BM25 didn't have it in its top 20.
- "Hoe rol ik een nieuwe versie eerst uit naar een klein deel van de gebruikers?" ("How do I roll out a new version to a small group of users first?") finds the canary deployment posts. BM25 can't match Dutch words against English posts.

And "How is this blog hosted?", the question from the previous post, now finds the post about the static site.

Vectors alone do almost as well, but embeddings are bad at names they don't know. For "Decision-1 gate", the best chunk of the post about the chat was at position 106 in the vector list, while BM25 had it first. Hybrid search keeps the exact match from BM25 and the paraphrases from the vectors.

⚠️ **Important:** I wrote these questions myself, knowing which posts answer them. I left out the words from the post titles on purpose, but 45 hand-written questions make a sanity check. A real benchmark needs questions from people who don't know the posts.

## So when do you need a vector database?

A vector database earns its place when:

- The collection is large. At hundreds of thousands of chunks, the memory and the scan time start to matter, and an approximate index (like HNSW or DiskANN) scans a fraction of the vectors.
- The data changes all the time. My vectors change when I publish a post, and a redeploy is fine. With content that changes every minute you want a store you can write to.
- You need filters and security. Search services handle "only documents this user may see, from the last year" well, and add things like semantic ranking.

Below a few tens of thousands of chunks, a flat file and a loop are hard to beat on simplicity and cost. Back in 2023 I [searched my blog posts with Pinecone](https://baeke.info/2023/03/16/pinecone-and-openai-magic-a-guide-to-finding-your-long-lost-blog-posts-with-vectorized-search-and-chatgpt/). I learned a lot from it, and a flat file would have done the job. 😎

## What you don't get

- Updates need a deploy, because the vectors ship with the function. A blog can live with that, a live catalog can't.
- Every question costs an extra call: hybrid search embeds it, which takes about 134 ms. If that call fails, the chat falls back to BM25.
- Every function instance holds all vectors in memory. 1.4 MB is nothing, a gigabyte is a different story.
- Metadata filters beyond "at most 2 chunks per post" would be code I write myself.

## Wrapping up

For a blog-sized collection, hybrid search fits in a 1.4 MB file and a loop that takes a third of a millisecond. On my test questions the chat now finds an answering post for 42 of 45 questions instead of 32.

The code lives in my private blog repo for now. You can try it yourself with the chat button at the bottom right. Ask it something in your own words, or in your own language: it finds the right posts and answers in English. 😊

If you spot any errors or have questions, feel free to reach out!