Skip to content
All tags

#bm25

5 posts
ai guide Ask AI in Practice

How Ask AI Finds Posts: Planner, Hybrid Retrieval, and Retry

Ask AI first extracts intent, complexity, and 1–4 search terms. It then routes across metadata, BM25, Vectorize, and RRF; a retry adds Critic gaps and disables the first-pass-only BM25 short circuit.

Why Did 'I Want a Beginner AI Course' Return Zero Results? Debugging Chinese Tokenization and a Broken RAG Data Path

Entering '我想找入門的ai課程' in Ask AI showed zero searched posts and triggered a refusal, while Related Reading recommended exactly the right article. The first fix addressed Chinese tokenization, the LIKE fallback, and the Vectorize data path. A second pass added short Han-and-number tokens, post metadata retrieval, and a rule that exposes sources only after both Validation and Critic pass.

When Search Returns Only 10 Results: Fixing CJK Recall in Cloudflare D1 FTS5 Hybrid Search

Querying “認證” returned only ~10 hits while 149 files (509 occurrences) matched; 41 posts and 76 chunks were found via LIKE, but D1 chunks_fts had 0 rows and unicode61/trigram both returned 0 for 2-char CJK terms. Fix: LIKE fallback with char-level OR first, then trigram migration + pnpm sync, then pagination beyond the hard limit of 12.

Hybrid Search: Using BM25 + Vector Search to Cover Each Other's Blind Spots

Vector search handles semantics; BM25 handles keywords. Combining them with RRF is what lets you handle both fuzzy queries and exact terms at the same time.

SPLADE: Smarter Sparse Vector Search Beyond BM25

BM25 only recognizes words that appear in the query. SPLADE infers related terms and adds them to the search, gaining partial semantic capability while preserving the precision of keyword search.