ajaynomics/context-kit
1
0
Fork 0
Code Issues Pull Requests Releases Activity
Files
be0f8ec5619738a4de5c09954cf9f04ba21eca34
context-kit/README.md
Ajay Krishnan 51dceee224 Overhaul docs retrieval and web search quality 
Replace the abandoned llms-txt-mcp/Chroma docs backend with an in-repo
MCP service: SQLite WAL + FTS5 + sentence-transformer embeddings,
transactional source replacement, persisted state across restarts,
singleflight refresh with conditional requests, hybrid lexical/semantic
ranking with exact-duplicate collapse, source/host filters, and
explicit-by-default content retrieval. Add docs_rebuild and a
docs-rebuild CLI command.

Add deterministic llms-full.txt snapshot generation for machine-local
menus with hash-validated provenance manifests; lifecycle commands
promote a local menu to its snapshot only when the manifest validates.
Switch public source profiles to content-bearing llms-full.txt feeds.

Improve web search: bounded provider fallback with per-attempt
diagnostics and cancellation, an optional Brave Search API provider,
strict SearXNG engine selection, capped link/media extraction, and a
real engine=browser renderer that routes every request through the
existing SSRF vetting while blocking WebSockets, non-GET traffic, and
private destinations.

Extend release checks with offline unit suites and isolated candidate
container tests for both images.
2026-07-25 08:49:26 -07:00

208 lines
6.9 KiB
Markdown
Raw Blame History

# Context Kit
Local context tools for Claude Code and OpenCode.
Local web search. Local docs. Repo packing. No API keys required.
## What You Get
Context Kit gives coding agents three local tools:
| Tool | Purpose |
|---|---|
| `context-web-search` | Current web search with fallback diagnostics plus safe HTTP/browser extraction |
| `context-docs` | Persisted hybrid lexical/semantic search over curated documentation |
| `context-repomix` | Pack repositories into AI-friendly context |
The first public release deliberately keeps the surface area small: web search,
docs search, and repository packing.
## Quick Start
```sh
git clone https://gitea.krishnan.ca/ajaynomics/context-kit.git
cd context-kit
cp .env.example .env
export PATH="$PWD/bin:$PATH"
bin/context-kit start
bin/context-kit doctor
```
Then connect your assistant.
For Claude Code:
```sh
bin/context-kit install claude
```
Copy the printed JSON into your project's `.mcp.json`, or use the equivalent
`claude mcp add` commands if you prefer managing servers through the Claude CLI.
The default snippet uses `context-kit` on `PATH`, which is the right shape for
shared project config. For a private user-only config, you can print absolute
paths with `bin/context-kit install claude --absolute`.
For OpenCode:
```sh
bin/context-kit install opencode
```
Merge the printed `mcp` block into your `opencode.json`, then restart OpenCode.
The default snippet uses `context-kit` on `PATH`. Use
`bin/context-kit install opencode --absolute` only for private, machine-local
config that will not be committed.
## How It Runs
- SearXNG binds to `127.0.0.1:8099` only.
- `context-web-search` and `context-docs` are long-lived HTTP MCP services on
`127.0.0.1:8777` and `127.0.0.1:8776`. All assistant projects reuse them.
- `bin/context-kit web-search` and `bin/context-kit docs` are stdio bridges for
local clients that cannot use HTTP MCP directly.
- `context-repomix` remains a per-call stdio container because its read-only
project mount is caller-specific.
- Shared containers and the network have Compose-derived deterministic names.
Web search records the owning host uid; client containers are named and
labeled per launcher process and remove only themselves.
- Web search uses stateless MCP HTTP sessions, validates Host, rejects every
supplied Origin, and exits for Docker restart if its stdio backend dies.
- Explicit `fetch_url engine=browser` renders JavaScript while routing every
network GET through the same DNS/private-address checks as HTTP fetching.
- `context-docs` browser CORS is disabled by default; set exact local origins
only when a browser-based client needs direct access.
- Docs use a transactional SQLite WAL/FTS5 index; docs and model caches live in
`$HOME/.local/share/context-kit` and survive container replacement.
- Docs refresh TTL defaults to `24h`.
- Repomix mounts only the current project read-only.
- No code-editing MCP server is enabled by default.
## Public Repo vs Local Runtime State
This repository is the public, portable Context Kit distribution. It should only
contain generic defaults, Docker/service code, install snippets, and optional
public docs source profiles.
Machine-specific configuration stays out of git:
- `.env` is ignored. Use it for local ports, data paths, and private source
profile paths.
- `CONTEXT_KIT_DATA_DIR` stores runtime state: docs indexes, model caches, the
generated `docs-sources.txt`, and any local source tree you choose to keep
there.
- Private docs source profiles can live anywhere outside the repo and can be
referenced from `.env` with an absolute path.
- Private `llms.txt` menus belong under `CONTEXT_KIT_DOCS_LOCAL_SOURCES_DIR`, not
under `config/`.
The public default is intentionally small: `config/sources.default.txt`. If a
machine adds extra local menus, they affect only that machine's running
`context-docs` service.
## Docs Sources
The default docs index uses the vendors' content-bearing `llms-full.txt` feeds:
- Claude Code docs
- OpenAI API docs and reference
- OpenRouter docs
- Model Context Protocol docs
Optional profiles live in `config/`:
- `sources.ruby-ai.txt`
- `sources.js.txt`
- `sources.cloudflare.txt`
Example:
```sh
CONTEXT_KIT_DOCS_SOURCES="config/sources.default.txt config/sources.js.txt" \
bin/context-kit restart
```
Source changes are loaded by `start`/`restart`; `bin/context-kit docs` is only a
stdio bridge to the already-running docs service.
`docs_query` searches with FTS5 plus embeddings, deduplicates exact content,
and supports source/host filters. It returns snippets but does not retrieve full
content unless IDs are requested or `auto_retrieve` is explicitly enabled.
`bin/context-kit docs-rebuild` safely replaces selected source generations only
after fetch, parse, and embedding succeed.
For machine-local menu files, `bin/context-kit docs-snapshot` fetches their
linked pages into deterministic sibling `llms-full.txt` files with a provenance
manifest and conditional-request cache. Lifecycle commands automatically prefer
that full snapshot while preserving a prior snapshot if regeneration fails.
Large vendor feeds are opt-in because they can expand to thousands of sections
and take a while to embed.
## Commands
```sh
bin/context-kit start
bin/context-kit stop
bin/context-kit build
bin/context-kit status
bin/context-kit doctor
bin/context-kit install claude
bin/context-kit install opencode
bin/context-kit redaction-check
bin/context-kit docs-snapshot
bin/context-kit docs-rebuild
```
MCP entrypoints:
```sh
bin/context-kit web-search
bin/context-kit docs
bin/context-kit repomix
```
For this upgrade from `origin/main`, build images and safely add the missing
shared web-search service without recreating the existing SearXNG or docs
containers:
```sh
bin/context-kit build
bin/context-kit start
```
`start` always uses Compose `--no-recreate`. `restart` restarts the same
container IDs and does not apply a rebuilt image or changed container
environment. Context Kit intentionally has no implicit destructive replacement
command.
When an update changes an MCP transport, regenerate the assistant snippet and
replace the corresponding configuration before restarting the assistant. The
current snippets connect both web search and docs directly over HTTP.
## Security Model
Context Kit is local-first, but MCP tools still extend what your agent can do.
- Treat fetched web pages as untrusted input.
- Do not expose SearXNG publicly without changing the secret and reviewing its
configuration.
- Keep docs profiles curated. More sources means more background indexing and
more untrusted text in your retrieval corpus.
- Be cautious when adding code-editing MCP servers. Context Kit's default MCP
servers either read remote content or mount the current project read-only.
See `docs/security.md` for details.
## Requirements
- Docker with Compose v2
- Bash
- `curl` for health checks
- `flock` from util-linux for serialized service lifecycle operations
No hosted API keys are required for the default stack.
## License
MIT
Reference in New Issue View Git Blame Copy Permalink