Overhaul docs retrieval and web search quality
Replace the abandoned llms-txt-mcp/Chroma docs backend with an in-repo MCP service: SQLite WAL + FTS5 + sentence-transformer embeddings, transactional source replacement, persisted state across restarts, singleflight refresh with conditional requests, hybrid lexical/semantic ranking with exact-duplicate collapse, source/host filters, and explicit-by-default content retrieval. Add docs_rebuild and a docs-rebuild CLI command. Add deterministic llms-full.txt snapshot generation for machine-local menus with hash-validated provenance manifests; lifecycle commands promote a local menu to its snapshot only when the manifest validates. Switch public source profiles to content-bearing llms-full.txt feeds. Improve web search: bounded provider fallback with per-attempt diagnostics and cancellation, an optional Brave Search API provider, strict SearXNG engine selection, capped link/media extraction, and a real engine=browser renderer that routes every request through the existing SSRF vetting while blocking WebSockets, non-GET traffic, and private destinations. Extend release checks with offline unit suites and isolated candidate container tests for both images.
This commit is contained in:
@@ -10,6 +10,11 @@ Context Kit is designed to be safe by default for local development.
|
||||
- The web-search MCP image runs as the non-root `node` user.
|
||||
- Web-search MCP sessions are stateless. Its HTTP front end permits only
|
||||
loopback/internal Host values and rejects every supplied Origin with 403.
|
||||
- Browser fetch intercepts each network GET, resolves it outside Chromium, and
|
||||
blocks private/localhost addresses, non-GET requests, request-count overflow,
|
||||
and byte-budget overflow. Redirect targets are checked independently.
|
||||
- Search diagnostics contain bounded categorized error messages and never emit
|
||||
the optional Brave credential.
|
||||
- Repomix mounts only the current project read-only.
|
||||
- Docs indexing stores data under `$HOME/.local/share/context-kit` unless you
|
||||
override it.
|
||||
@@ -27,6 +32,11 @@ Only index sources you trust enough to retrieve into an agent conversation. More
|
||||
sources are not always better. Large or noisy docs can make retrieval slower and
|
||||
less precise.
|
||||
|
||||
Docs source replacement is transactional. SQLite WAL state persists on the docs
|
||||
volume, removed source profiles become inactive immediately, and full content is
|
||||
not returned by default. Local snapshot provenance is stored separately from the
|
||||
retrieval text so metadata does not pollute ranking.
|
||||
|
||||
## Code-Editing MCP Servers
|
||||
|
||||
Context Kit's default MCP servers either read remote content or mount the
|
||||
|
||||
Reference in New Issue
Block a user