ajaynomics/context-kit
1
0
Fork 0
Code Issues Pull Requests Actions Releases Activity

Simplify runtime checks and MCP smokes

Browse Source
This commit is contained in:
Ajay Krishnan
2026-06-25 09:19:26 -07:00
parent 99881b608b
commit 8da552bea1
13 changed files with 476 additions and 465 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
README.md
View File

@@ -6,13 +6,13 @@ Local web search. Local docs. Repo packing. No API keys required.
## What You Get ## What You Get
Context Kit gives coding agents three local MCP servers: Context Kit gives coding agents three local tools:
| Server | Purpose | Default | | Tool | Purpose |
|---|---|---| |---|---|
| `context-web-search` | Current web search through local SearXNG plus URL fetch/extract | Enabled | | `context-web-search` | Current web search through local SearXNG plus URL fetch/extract |
| `context-docs` | Semantic search over curated `llms.txt` documentation | Enabled | | `context-docs` | Semantic search over curated `llms.txt` documentation |
| `context-repomix` | Pack local or remote repositories into AI-friendly context | Enabled | | `context-repomix` | Pack repositories into AI-friendly context |
The first public release deliberately keeps the surface area small: web search, The first public release deliberately keeps the surface area small: web search,
docs search, and repository packing. docs search, and repository packing.
@@ -53,23 +53,17 @@ The default snippet uses `context-kit` on `PATH`. Use
`bin/context-kit install opencode --absolute` only for private, machine-local `bin/context-kit install opencode --absolute` only for private, machine-local
config that will not be committed. config that will not be committed.
## Defaults ## How It Runs
- SearXNG binds to `127.0.0.1:8099` only. - SearXNG binds to `127.0.0.1:8099` only.
- `context-web-search` defaults `search_web` to SearXNG, then falls back to - `context-web-search` and `context-repomix` run as local stdio MCP commands.
DuckDuckGo and Bing. Bing uses Chromium inside the web-search image. - `context-docs` runs as a local HTTP MCP service. `bin/context-kit docs` is a
- `fetch_url` uses upstream HTTP extraction. In `mcp-web-search` 1.3.0, stdio fallback for clients that cannot use HTTP MCP.
`engine=browser` is accepted but does not invoke Chromium yet.
- `context-docs` runs as a long-lived service on `127.0.0.1:8776` (Streamable
HTTP MCP) so every client shares one indexer and one Chroma writer. The
`bin/context-kit docs` stdio command is kept as a compatibility shim for
clients that cannot speak HTTP MCP.
- `context-docs` browser CORS is disabled by default; set exact local origins - `context-docs` browser CORS is disabled by default; set exact local origins
only when a browser-based client needs direct access. only when a browser-based client needs direct access.
- Docs and model caches live in `$HOME/.local/share/context-kit`. - Docs and model caches live in `$HOME/.local/share/context-kit`.
- Docs refresh TTL defaults to `24h`. - Docs refresh TTL defaults to `24h`.
- MCP containers are labeled `dev.context-kit=true` for safe inspection and cleanup. - Repomix mounts only the current project read-only.
- Repomix mounts only the current project read-only, not your whole home directory.
- No code-editing MCP server is enabled by default. - No code-editing MCP server is enabled by default.
## Docs Sources ## Docs Sources
@@ -91,9 +85,12 @@ Example:
```sh ```sh
CONTEXT_KIT_DOCS_SOURCES="config/sources.default.txt config/sources.js.txt" \ CONTEXT_KIT_DOCS_SOURCES="config/sources.default.txt config/sources.js.txt" \
bin/context-kit docs bin/context-kit restart
``` ```
Source changes are loaded by `start`/`restart`; `bin/context-kit docs` is only a
stdio bridge to the already-running docs service.
Large vendor feeds are opt-in because they can expand to thousands of sections Large vendor feeds are opt-in because they can expand to thousands of sections
and take a while to embed. and take a while to embed.
@@ -105,6 +102,8 @@ bin/context-kit stop
bin/context-kit build bin/context-kit build
bin/context-kit status bin/context-kit status
bin/context-kit doctor bin/context-kit doctor
bin/context-kit install claude
bin/context-kit install opencode
bin/context-kit redaction-check bin/context-kit redaction-check
``` ```
@@ -116,6 +115,13 @@ bin/context-kit docs
bin/context-kit repomix bin/context-kit repomix
``` ```
After pulling Context Kit updates, rebuild local images and restart services:
```sh
bin/context-kit build
bin/context-kit restart
```
## Security Model ## Security Model
Context Kit is local-first, but MCP tools still extend what your agent can do. Context Kit is local-first, but MCP tools still extend what your agent can do.

146
bin/context-kit
View File

@@ -42,22 +42,26 @@ fail() {
load_env_file load_env_file
DEFAULT_DATA_DIR="${HOME:-${PWD}}/.local/share/context-kit" if [[ -z "${CONTEXT_KIT_DATA_DIR:-}" && -z "${HOME:-}" ]]; then
fail "HOME or CONTEXT_KIT_DATA_DIR must be set"
fi
DEFAULT_DATA_DIR="${HOME:-}/.local/share/context-kit"
PROJECT="${CONTEXT_KIT_COMPOSE_PROJECT:-context-kit}" PROJECT="${CONTEXT_KIT_COMPOSE_PROJECT:-context-kit}"
COMPOSE_FILE="${ROOT}/compose.yml" COMPOSE_FILE="${ROOT}/compose.yml"
DATA_DIR="${CONTEXT_KIT_DATA_DIR:-${DEFAULT_DATA_DIR}}" DATA_DIR="${CONTEXT_KIT_DATA_DIR:-${DEFAULT_DATA_DIR}}"
NETWORK="${CONTEXT_KIT_DOCKER_NETWORK:-${PROJECT}_default}" NETWORK="${PROJECT}_default"
SEARXNG_PORT="${CONTEXT_KIT_SEARXNG_PORT:-8099}" SEARXNG_PORT="${CONTEXT_KIT_SEARXNG_PORT:-8099}"
DOCS_PORT="${CONTEXT_KIT_DOCS_PORT:-8776}" DOCS_PORT="${CONTEXT_KIT_DOCS_PORT:-8776}"
DOCS_HTTP_URL="${CONTEXT_KIT_DOCS_HTTP_URL:-http://127.0.0.1:${DOCS_PORT}/mcp}" DOCS_HTTP_URL="${CONTEXT_KIT_DOCS_HTTP_URL:-http://127.0.0.1:${DOCS_PORT}/mcp}"
WEB_SEARCH_MAX_BYTES="${CONTEXT_KIT_WEB_SEARCH_MAX_BYTES:-52428800}" WEB_SEARCH_MAX_BYTES="${CONTEXT_KIT_WEB_SEARCH_MAX_BYTES:-52428800}"
WEB_SEARCH_PROVIDER="${CONTEXT_KIT_WEB_SEARCH_PROVIDER:-${DEFAULT_SEARCH_PROVIDER:-searxng}}" WEB_SEARCH_PROVIDER="${CONTEXT_KIT_WEB_SEARCH_PROVIDER:-searxng}"
WEB_SEARCH_HTTP_TIMEOUT="${CONTEXT_KIT_WEB_SEARCH_HTTP_TIMEOUT:-${HTTP_TIMEOUT:-15000}}" WEB_SEARCH_HTTP_TIMEOUT="${CONTEXT_KIT_WEB_SEARCH_HTTP_TIMEOUT:-15000}"
WEB_SEARCH_MAX_RESULTS="${CONTEXT_KIT_WEB_SEARCH_MAX_RESULTS:-${MAX_RESULTS:-10}}" WEB_SEARCH_MAX_RESULTS="${CONTEXT_KIT_WEB_SEARCH_MAX_RESULTS:-10}"
WEB_SEARCH_CHROME_PATH="${CONTEXT_KIT_WEB_SEARCH_CHROME_PATH:-${CHROME_PATH:-/usr/bin/chromium}}" WEB_SEARCH_CHROME_PATH="${CONTEXT_KIT_WEB_SEARCH_CHROME_PATH:-/usr/bin/chromium}"
WEB_SEARCH_BROWSER_USER_AGENT="${CONTEXT_KIT_WEB_SEARCH_BROWSER_USER_AGENT:-${BROWSER_SEARCH_USER_AGENT:-}}" WEB_SEARCH_BROWSER_USER_AGENT="${CONTEXT_KIT_WEB_SEARCH_BROWSER_USER_AGENT:-}"
WEB_SEARCH_MCP_COMPAT_MODE="${CONTEXT_KIT_WEB_SEARCH_MCP_COMPAT_MODE:-${MCP_COMPAT_MODE:-}}" WEB_SEARCH_MCP_COMPAT_MODE="${CONTEXT_KIT_WEB_SEARCH_MCP_COMPAT_MODE:-}"
DOCS_CONTAINER_NAME="context-kit-docs-mcp" DOCS_SERVICE_NAME="docs-mcp"
DOCS_SOURCES_FILE="${DATA_DIR}/docs-sources.txt" DOCS_SOURCES_FILE="${DATA_DIR}/docs-sources.txt"
DOCS_DATA_DIR="${DATA_DIR}/docs" DOCS_DATA_DIR="${DATA_DIR}/docs"
MODELS_DATA_DIR="${DATA_DIR}/models" MODELS_DATA_DIR="${DATA_DIR}/models"
@@ -108,10 +112,18 @@ compose() {
CONTEXT_KIT_DOCS_PREINDEX="${CONTEXT_KIT_DOCS_PREINDEX:-0}" \ CONTEXT_KIT_DOCS_PREINDEX="${CONTEXT_KIT_DOCS_PREINDEX:-0}" \
CONTEXT_KIT_DOCS_LOCAL_SOURCES_DIR="${DOCS_LOCAL_SOURCES_DIR}" \ CONTEXT_KIT_DOCS_LOCAL_SOURCES_DIR="${DOCS_LOCAL_SOURCES_DIR}" \
CONTEXT_KIT_DOCS_LOCAL_SOURCES_PORT="${DOCS_LOCAL_SOURCES_PORT}" \ CONTEXT_KIT_DOCS_LOCAL_SOURCES_PORT="${DOCS_LOCAL_SOURCES_PORT}" \
BUILDX_BUILDER="${CONTEXT_KIT_BUILDX_BUILDER:-${BUILDX_BUILDER:-default}}" \ CONTEXT_KIT_WEB_SEARCH_IMAGE="${WEB_SEARCH_IMAGE}" \
CONTEXT_KIT_DOCS_IMAGE="${DOCS_IMAGE}" \
BUILDX_BUILDER="${CONTEXT_KIT_BUILDX_BUILDER:-default}" \
docker compose -p "${PROJECT}" -f "${COMPOSE_FILE}" "$@" docker compose -p "${PROJECT}" -f "${COMPOSE_FILE}" "$@"
} }
require_no_args() {
local usage_text="$1"
shift
[[ "$#" -eq 0 ]] || fail "${usage_text}"
}
write_docs_sources_file() { write_docs_sources_file() {
mkdir -p "$(dirname "${DOCS_SOURCES_FILE}")" mkdir -p "$(dirname "${DOCS_SOURCES_FILE}")"
local tmp="${DOCS_SOURCES_FILE}.tmp.$$" local tmp="${DOCS_SOURCES_FILE}.tmp.$$"
@@ -152,41 +164,6 @@ check_data_dirs() {
return "${ok}" return "${ok}"
} }
check_web_search_schema_patch() {
docker run --rm --entrypoint node \
-e MAX_BYTES="${WEB_SEARCH_MAX_BYTES}" \
-e EXPECTED_MAX_BYTES="${WEB_SEARCH_MAX_BYTES}" \
"${WEB_SEARCH_IMAGE}" \
-e '
const fs = require("node:fs");
const expected = Number(process.env.EXPECTED_MAX_BYTES) || 0;
const actual = Number(process.env.MAX_BYTES) || 0;
const serverPath = "/usr/local/lib/node_modules/@zhafron/mcp-web-search/dist/src/server.js";
const source = fs.readFileSync(serverPath, "utf8");
if (actual !== expected) process.exit(1);
if (!source.includes("max_download_bytes: z.number().int().min(1).max(MAX_BYTES).optional()")) process.exit(1);
' >/dev/null 2>&1
}
check_web_search_bing_override() {
docker run --rm --entrypoint node \
"${WEB_SEARCH_IMAGE}" \
-e '
const fs = require("node:fs");
const bingPath = "/usr/local/lib/node_modules/@zhafron/mcp-web-search/dist/src/providers/bing.js";
const source = fs.readFileSync(bingPath, "utf8");
if (!source.includes("Context Kit override for @zhafron/mcp-web-search 1.3.0")) process.exit(1);
if (!source.includes("waitForSelector")) process.exit(1);
if (!source.includes("decodeBingRedirect")) process.exit(1);
' >/dev/null 2>&1
}
check_web_search_chrome() {
docker run --rm --entrypoint /usr/bin/test \
"${WEB_SEARCH_IMAGE}" \
-x "${WEB_SEARCH_CHROME_PATH}" >/dev/null 2>&1
}
warn() { warn() {
printf 'warn: %s\n' "$*" >&2 printf 'warn: %s\n' "$*" >&2
} }
@@ -239,21 +216,35 @@ wait_for_searxng() {
done done
warn "SearXNG did not become ready on 127.0.0.1:${SEARXNG_PORT} after 30s" warn "SearXNG did not become ready on 127.0.0.1:${SEARXNG_PORT} after 30s"
return 1
}
docs_service_running() {
local container_id
container_id="$(compose ps -q "${DOCS_SERVICE_NAME}" 2>/dev/null || true)"
[[ -n "${container_id}" ]] || return 1
docker inspect -f '{{.State.Running}}' "${container_id}" 2>/dev/null | grep -qx true
} }
wait_for_docs_mcp() { wait_for_docs_mcp() {
command -v curl >/dev/null 2>&1 || return 0 command -v curl >/dev/null 2>&1 || return 0
# First-run can take a while: model download + full preindex of every source. # First run can take a while: model download plus optional eager preindexing.
local attempt local attempt http_ready=0
for attempt in {1..180}; do for attempt in {1..180}; do
if curl -fsS -o /dev/null "http://127.0.0.1:${DOCS_PORT}/status" 2>/dev/null; then if curl -fsS -o /dev/null "http://127.0.0.1:${DOCS_PORT}/status" 2>/dev/null; then
return 0 http_ready=1
break
fi fi
sleep 1 sleep 1
done done
warn "docs-mcp did not become ready on 127.0.0.1:${DOCS_PORT} after 180s (check: docker logs ${DOCS_CONTAINER_NAME})" if [[ "${http_ready}" -ne 1 ]]; then
warn "docs-mcp did not become ready on 127.0.0.1:${DOCS_PORT} after 180s (check: docker compose logs ${DOCS_SERVICE_NAME})"
return 1
fi
return 0
} }
abs_dir() { abs_dir() {
@@ -293,7 +284,7 @@ resolved_sources() {
} }
cmd_build() { cmd_build() {
[[ "$#" -eq 0 ]] || fail "usage: context-kit build" require_no_args "usage: context-kit build" "$@"
require_docker require_docker
# web-search-mcp is still profile-gated (built but not auto-started); # web-search-mcp is still profile-gated (built but not auto-started);
# docs-mcp is a regular long-lived service so it builds without a profile. # docs-mcp is a regular long-lived service so it builds without a profile.
@@ -303,6 +294,7 @@ cmd_build() {
} }
cmd_start() { cmd_start() {
require_no_args "usage: context-kit start" "$@"
require_docker require_docker
prepare_data_dirs prepare_data_dirs
if ! docker image inspect "${WEB_SEARCH_IMAGE}" >/dev/null 2>&1 || ! docker image inspect "${DOCS_IMAGE}" >/dev/null 2>&1; then if ! docker image inspect "${WEB_SEARCH_IMAGE}" >/dev/null 2>&1 || ! docker image inspect "${DOCS_IMAGE}" >/dev/null 2>&1; then
@@ -315,11 +307,19 @@ cmd_start() {
} }
cmd_stop() { cmd_stop() {
require_no_args "usage: context-kit stop" "$@"
require_docker require_docker
compose stop searxng docs-mcp compose stop searxng docs-mcp
} }
cmd_restart() {
require_no_args "usage: context-kit restart" "$@"
cmd_stop
cmd_start
}
cmd_status() { cmd_status() {
require_no_args "usage: context-kit status" "$@"
require_docker require_docker
printf 'Services\n' printf 'Services\n'
compose ps compose ps
@@ -327,9 +327,9 @@ cmd_status() {
docker image ls --format '{{.Repository}}:{{.Tag}}\t{{.Size}}' \ docker image ls --format '{{.Repository}}:{{.Tag}}\t{{.Size}}' \
| grep -E '^(context-kit/|ghcr.io/yamadashy/repomix:)' || true | grep -E '^(context-kit/|ghcr.io/yamadashy/repomix:)' || true
printf '\nActive per-call MCP containers\n' printf '\nActive per-call MCP containers\n'
docker ps -a --filter label=dev.context-kit=true --format '{{.Names}}\t{{.Status}}\t{{.Image}}\t{{.Command}}' \ docker ps -a --filter label=dev.context-kit=true --format '{{.Names}}\t{{.Status}}\t{{.Image}}\t{{.Command}}\t{{.Label "com.docker.compose.service"}}' \
| awk 'BEGIN { print "NAMES\tSTATUS\tIMAGE\tCOMMAND" } $1 !~ /^context-kit-(docs-mcp|searxng-1)$/ { print }' | awk -F '\t' 'BEGIN { print "NAMES\tSTATUS\tIMAGE\tCOMMAND" } $5 !~ /^(searxng|docs-mcp)$/ { print $1 "\t" $2 "\t" $3 "\t" $4 }'
printf '\nDocs MCP endpoint\n- %s (container: %s)\n' "${DOCS_HTTP_URL}" "${DOCS_CONTAINER_NAME}" printf '\nDocs MCP endpoint\n- %s (service: %s)\n' "${DOCS_HTTP_URL}" "${DOCS_SERVICE_NAME}"
printf '\nDocs sources\n' printf '\nDocs sources\n'
resolved_sources | sed 's/^/- /' resolved_sources | sed 's/^/- /'
printf '\nLocal docs source directory\n- %s (served inside docs-mcp at http://127.0.0.1:%s/)\n' "${DOCS_LOCAL_SOURCES_DIR}" "${DOCS_LOCAL_SOURCES_PORT}" printf '\nLocal docs source directory\n- %s (served inside docs-mcp at http://127.0.0.1:%s/)\n' "${DOCS_LOCAL_SOURCES_DIR}" "${DOCS_LOCAL_SOURCES_PORT}"
@@ -337,6 +337,7 @@ cmd_status() {
} }
cmd_doctor() { cmd_doctor() {
require_no_args "usage: context-kit doctor" "$@"
local ok=0 local ok=0
printf 'Context Kit doctor\n' printf 'Context Kit doctor\n'
@@ -376,27 +377,6 @@ cmd_doctor() {
fi fi
done done
if docker image inspect "${WEB_SEARCH_IMAGE}" >/dev/null 2>&1; then
if check_web_search_schema_patch; then
printf 'pass web-search fetch_url max-bytes schema patch: %s\n' "${WEB_SEARCH_MAX_BYTES}"
else
printf 'fail web-search max-bytes schema patch missing; run: context-kit build\n'
ok=1
fi
if check_web_search_bing_override; then
printf 'pass web-search Bing provider override installed\n'
else
printf 'fail web-search Bing provider override missing; run: context-kit build\n'
ok=1
fi
if check_web_search_chrome; then
printf 'pass web-search Chromium path: %s\n' "${WEB_SEARCH_CHROME_PATH}"
else
printf 'fail web-search Chromium path unavailable: %s\n' "${WEB_SEARCH_CHROME_PATH}"
ok=1
fi
fi
if command -v curl >/dev/null 2>&1 && curl -fsS "http://127.0.0.1:${SEARXNG_PORT}/healthz" >/dev/null 2>&1; then if command -v curl >/dev/null 2>&1 && curl -fsS "http://127.0.0.1:${SEARXNG_PORT}/healthz" >/dev/null 2>&1; then
printf 'pass SearXNG responds on 127.0.0.1:%s\n' "${SEARXNG_PORT}" printf 'pass SearXNG responds on 127.0.0.1:%s\n' "${SEARXNG_PORT}"
else else
@@ -421,6 +401,7 @@ cmd_doctor() {
} }
cmd_web_search() { cmd_web_search() {
require_no_args "usage: context-kit web-search" "$@"
require_docker require_docker
require_network require_network
require_image "${WEB_SEARCH_IMAGE}" "context-kit build" require_image "${WEB_SEARCH_IMAGE}" "context-kit build"
@@ -433,7 +414,7 @@ cmd_web_search() {
"${cidfile_args[@]}" \ "${cidfile_args[@]}" \
--network "${NETWORK}" \ --network "${NETWORK}" \
-e DEFAULT_SEARCH_PROVIDER="${WEB_SEARCH_PROVIDER}" \ -e DEFAULT_SEARCH_PROVIDER="${WEB_SEARCH_PROVIDER}" \
-e SEARXNG_URL="${SEARXNG_URL:-http://searxng:8080}" \ -e SEARXNG_URL="http://searxng:8080" \
-e CHROME_PATH="${WEB_SEARCH_CHROME_PATH}" \ -e CHROME_PATH="${WEB_SEARCH_CHROME_PATH}" \
-e HTTP_TIMEOUT="${WEB_SEARCH_HTTP_TIMEOUT}" \ -e HTTP_TIMEOUT="${WEB_SEARCH_HTTP_TIMEOUT}" \
-e MAX_BYTES="${WEB_SEARCH_MAX_BYTES}" \ -e MAX_BYTES="${WEB_SEARCH_MAX_BYTES}" \
@@ -444,6 +425,7 @@ cmd_web_search() {
} }
cmd_docs() { cmd_docs() {
require_no_args "usage: context-kit docs" "$@"
# Prefer the `type: remote` MCP config pointing at ${DOCS_HTTP_URL}. # Prefer the `type: remote` MCP config pointing at ${DOCS_HTTP_URL}.
# This stdio entrypoint is kept for clients that cannot speak HTTP MCP: # This stdio entrypoint is kept for clients that cannot speak HTTP MCP:
# it spawns a thin mcp-proxy bridge per call but all calls multiplex onto # it spawns a thin mcp-proxy bridge per call but all calls multiplex onto
@@ -453,11 +435,11 @@ cmd_docs() {
require_network require_network
require_image "${DOCS_IMAGE}" "context-kit build" require_image "${DOCS_IMAGE}" "context-kit build"
if ! docker ps --filter "name=^${DOCS_CONTAINER_NAME}$" --filter "status=running" --format '{{.Names}}' | grep -qx "${DOCS_CONTAINER_NAME}"; then if ! docs_service_running; then
fail "long-lived docs-mcp not running; start it with: context-kit start" fail "long-lived docs-mcp not running; start it with: context-kit start"
fi fi
local bridge_url="http://${DOCS_CONTAINER_NAME}:8000/mcp" local bridge_url="http://${DOCS_SERVICE_NAME}:8000/mcp"
local cidfile_args=() local cidfile_args=()
if [[ -n "${CONTEXT_KIT_DOCKER_CIDFILE:-}" ]]; then if [[ -n "${CONTEXT_KIT_DOCKER_CIDFILE:-}" ]]; then
cidfile_args=(--cidfile "${CONTEXT_KIT_DOCKER_CIDFILE}") cidfile_args=(--cidfile "${CONTEXT_KIT_DOCKER_CIDFILE}")
@@ -473,6 +455,7 @@ cmd_docs() {
} }
cmd_repomix() { cmd_repomix() {
require_no_args "usage: context-kit repomix" "$@"
require_docker require_docker
require_image "${REPOMIX_IMAGE}" "docker pull ${REPOMIX_IMAGE}" require_image "${REPOMIX_IMAGE}" "docker pull ${REPOMIX_IMAGE}"
local dir mount_dir local dir mount_dir
@@ -557,9 +540,12 @@ JSON
cmd_install() { cmd_install() {
local target="${1:-}" local target="${1:-}"
shift || true shift || true
local option="${1:-}"
shift || true
[[ "$#" -eq 0 ]] || fail "usage: context-kit install claude|opencode [--absolute]"
case "${target}" in case "${target}" in
opencode) print_opencode "${1:-}" ;; opencode) print_opencode "${option}" ;;
claude) print_claude "${1:-}" ;; claude) print_claude "${option}" ;;
*) fail "usage: context-kit install claude|opencode [--absolute]" ;; *) fail "usage: context-kit install claude|opencode [--absolute]" ;;
esac esac
} }
@@ -611,7 +597,7 @@ cmd_redaction_check() {
case "${1:-}" in case "${1:-}" in
start) shift; cmd_start "$@" ;; start) shift; cmd_start "$@" ;;
stop) shift; cmd_stop "$@" ;; stop) shift; cmd_stop "$@" ;;
restart) shift; cmd_stop; cmd_start "$@" ;; restart) shift; cmd_restart "$@" ;;
build) shift; cmd_build "$@" ;; build) shift; cmd_build "$@" ;;
status) shift; cmd_status "$@" ;; status) shift; cmd_status "$@" ;;
doctor) shift; cmd_doctor "$@" ;; doctor) shift; cmd_doctor "$@" ;;

5
compose.yml
View File

@@ -20,7 +20,7 @@ services:
context: ./docker/web-search context: ./docker/web-search
args: args:
MCP_WEB_SEARCH_MAX_BYTES: "${CONTEXT_KIT_WEB_SEARCH_MAX_BYTES:-52428800}" MCP_WEB_SEARCH_MAX_BYTES: "${CONTEXT_KIT_WEB_SEARCH_MAX_BYTES:-52428800}"
image: context-kit/web-search-mcp:latest image: ${CONTEXT_KIT_WEB_SEARCH_IMAGE:-context-kit/web-search-mcp:latest}
profiles: ["mcp"] profiles: ["mcp"]
stdin_open: true stdin_open: true
tty: false tty: false
@@ -39,10 +39,9 @@ services:
docs-mcp: docs-mcp:
build: build:
context: ./docker/docs context: ./docker/docs
image: context-kit/docs-mcp:latest image: ${CONTEXT_KIT_DOCS_IMAGE:-context-kit/docs-mcp:latest}
# Long-lived shared docs MCP. One container = one Chroma writer; clients # Long-lived shared docs MCP. One container = one Chroma writer; clients
# connect over Streamable HTTP (mcp-proxy bridges llms-txt-mcp's stdio). # connect over Streamable HTTP (mcp-proxy bridges llms-txt-mcp's stdio).
container_name: context-kit-docs-mcp
restart: unless-stopped restart: unless-stopped
ports: ports:
- "127.0.0.1:${CONTEXT_KIT_DOCS_PORT:-8776}:8000" - "127.0.0.1:${CONTEXT_KIT_DOCS_PORT:-8776}:8000"

6
docker/docs/entrypoint.sh
View File

@@ -15,6 +15,12 @@ sources_file="${DOCS_MCP_SOURCES_FILE:-/etc/context-kit/docs-sources.txt}"
local_sources_dir="${DOCS_MCP_LOCAL_SOURCES_DIR:-/etc/context-kit/local-sources}" local_sources_dir="${DOCS_MCP_LOCAL_SOURCES_DIR:-/etc/context-kit/local-sources}"
local_sources_port="${DOCS_MCP_LOCAL_SOURCES_PORT:-8769}" local_sources_port="${DOCS_MCP_LOCAL_SOURCES_PORT:-8769}"
if [ ! -f "$sources_file" ]; then
echo "docs-mcp: sources file missing: $sources_file" >&2
echo "docs-mcp: run bin/context-kit start to generate it, or mount a file at that path." >&2
exit 64
fi
if [ ! -r "$sources_file" ]; then if [ ! -r "$sources_file" ]; then
echo "docs-mcp: sources file not readable: $sources_file" >&2 echo "docs-mcp: sources file not readable: $sources_file" >&2
echo "docs-mcp: set DOCS_MCP_SOURCES_FILE or mount one at that path." >&2 echo "docs-mcp: set DOCS_MCP_SOURCES_FILE or mount one at that path." >&2

6
docker/web-search/Dockerfile
View File

@@ -24,9 +24,13 @@ RUN npm install -g "@zhafron/mcp-web-search@${MCP_WEB_SEARCH_VERSION}" \
ENV CHROME_PATH=/usr/bin/chromium \ ENV CHROME_PATH=/usr/bin/chromium \
DEFAULT_SEARCH_PROVIDER=searxng \ DEFAULT_SEARCH_PROVIDER=searxng \
HOME=/tmp \
HTTP_TIMEOUT=15000 \ HTTP_TIMEOUT=15000 \
MAX_BYTES=${MCP_WEB_SEARCH_MAX_BYTES} \ MAX_BYTES=${MCP_WEB_SEARCH_MAX_BYTES} \
MAX_RESULTS=10 \ MAX_RESULTS=10 \
SEARXNG_URL=http://searxng:8080 SEARXNG_URL=http://searxng:8080 \
XDG_CACHE_HOME=/tmp/.cache
USER node
ENTRYPOINT ["mcp-web-search"] ENTRYPOINT ["mcp-web-search"]

11
docs/assistants.md
View File

@@ -1,7 +1,14 @@
# Assistant Setup # Assistant Setup
Context Kit supports any assistant that can run local stdio MCP servers. The Context Kit supports assistants that can run local stdio MCP servers, HTTP MCP
included snippets cover Claude Code and OpenCode. servers, or both. The default transport split is simple:
- `context-web-search`: local stdio command.
- `context-docs`: local HTTP MCP service.
- `context-repomix`: local stdio command.
`bin/context-kit docs` is a stdio fallback for clients that cannot use HTTP MCP.
The included snippets cover Claude Code and OpenCode.
## Claude Code ## Claude Code

18
docs/configuration.md
View File

@@ -7,7 +7,10 @@ Explicit environment variables win over `.env` values. The `.env` parser accepts
simple `KEY=VALUE` lines for `CONTEXT_KIT_*` variables only; it does not execute simple `KEY=VALUE` lines for `CONTEXT_KIT_*` variables only; it does not execute
shell code. shell code.
## Core Variables ## User-Facing Variables
Only the variables below are part of the public configuration surface. Other
`CONTEXT_KIT_*` variables used by scripts are release/test hooks and may change.
| Variable | Default | Purpose | | Variable | Default | Purpose |
|---|---|---| |---|---|---|
@@ -68,14 +71,21 @@ Avoid `*`; the docs MCP is a local unauthenticated endpoint.
## Source Profiles ## Source Profiles
The docs MCP accepts one or more source files: The docs MCP accepts one or more source profile files:
```sh ```sh
CONTEXT_KIT_DOCS_SOURCES="config/sources.default.txt config/sources.js.txt" CONTEXT_KIT_DOCS_SOURCES="config/sources.default.txt config/sources.js.txt"
``` ```
Each source file is plain text. Blank lines and `#` comments are ignored. Source changes are loaded when the docs service starts. Run `bin/context-kit
Entries may be absolute source-profile paths for private machine-local config. restart` after changing `CONTEXT_KIT_DOCS_SOURCES`; `bin/context-kit docs` only
bridges stdio clients to the already-running service.
`CONTEXT_KIT_DOCS_SOURCES` may include absolute paths to private machine-local
profile files. Each profile file is plain text; blank lines and `#` comments are
ignored. Entries inside profile files must be URLs ending in `/llms.txt` or
`/llms-full.txt`.
For local llms.txt files, place content under For local llms.txt files, place content under
`CONTEXT_KIT_DOCS_LOCAL_SOURCES_DIR` and reference it as `CONTEXT_KIT_DOCS_LOCAL_SOURCES_DIR` and reference it as
`http://127.0.0.1:8769/path/inside/local-sources/llms.txt` or another URL that `http://127.0.0.1:8769/path/inside/local-sources/llms.txt` or another URL that

1
docs/security.md
View File

@@ -6,6 +6,7 @@ Context Kit is designed to be safe by default for local development.
- SearXNG is bound to `127.0.0.1` only. - SearXNG is bound to `127.0.0.1` only.
- No hosted API keys are required. - No hosted API keys are required.
- The web-search MCP image runs as the non-root `node` user.
- Repomix mounts only the current project read-only. - Repomix mounts only the current project read-only.
- Docs indexing stores data under `$HOME/.local/share/context-kit` unless you - Docs indexing stores data under `$HOME/.local/share/context-kit` unless you
override it. override it.

29
docs/troubleshooting.md
View File

@@ -6,8 +6,21 @@
bin/context-kit doctor bin/context-kit doctor
``` ```
This checks Docker, Compose, images, the Docker network, SearXNG health, and This checks Docker, Compose, images, the Docker network, SearXNG health, docs
docs source configuration. HTTP readiness, and docs source configuration.
For release-grade MCP protocol checks, run:
```sh
scripts/release-check
```
Live provider checks are opt-in because search engines, remote docs, and model
downloads can fail independently of this repo:
```sh
CONTEXT_KIT_LIVE_CHECKS=1 scripts/release-check
```
## SearXNG Is Not Responding ## SearXNG Is Not Responding
@@ -66,11 +79,13 @@ URL fetching uses the HTTP extractor path.
## Docs Indexing Is Slow ## Docs Indexing Is Slow
The first run downloads an embedding model and embeds every configured docs The first `docs_query` or `docs_refresh` downloads an embedding model and
section. Keep default sources small, and add profiles only when you need them. embeds the requested docs sections lazily. Keep default sources small, and add
profiles only when you need them.
Cloudflare and other large docs sets can take significantly longer than the Cloudflare and other large docs sets can take significantly longer than the
default source profile. default source profile. Set `CONTEXT_KIT_DOCS_PREINDEX=1` only if you want
startup to eagerly embed every configured source.
## Docs Tools Say Index Manager Not Initialized ## Docs Tools Say Index Manager Not Initialized
@@ -79,7 +94,7 @@ If `docs_query` or `docs_refresh` returns `Index manager not initialized` while
initialize its embedding model or Chroma database. Check the container logs: initialize its embedding model or Chroma database. Check the container logs:
```sh ```sh
docker logs context-kit-docs-mcp docker compose -p "${CONTEXT_KIT_COMPOSE_PROJECT:-context-kit}" -f compose.yml logs docs-mcp
``` ```
A common cause is Docker creating the bind-mounted cache directories as `root` A common cause is Docker creating the bind-mounted cache directories as `root`
@@ -93,7 +108,7 @@ unable to open database file
Fix ownership and restart: Fix ownership and restart:
```sh ```sh
DATA_DIR="${CONTEXT_KIT_DATA_DIR:-$HOME/.local/share/context-kit}" DATA_DIR="${CONTEXT_KIT_DATA_DIR:-${HOME:?Set HOME or CONTEXT_KIT_DATA_DIR}/.local/share/context-kit}"
sudo chown -R "$(id -u):$(id -g)" "$DATA_DIR/docs" "$DATA_DIR/models" sudo chown -R "$(id -u):$(id -g)" "$DATA_DIR/docs" "$DATA_DIR/models"
bin/context-kit restart bin/context-kit restart
``` ```

188
scripts/mcp-smoke-client.mjs Normal file
View File

@@ -0,0 +1,188 @@
import { spawn, spawnSync } from "node:child_process";
import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
export function textFrom(result) {
return (result.content || [])
.filter(part => part.type === "text")
.map(part => part.text)
.join("\n");
}
export function requireToolSuccess(name, result) {
if (result?.isError) {
const payload = textFrom(result) || JSON.stringify(result);
throw new Error(`${name} returned an error: ${payload.slice(0, 500)}`);
}
return result;
}
export async function runSmoke({ usage, tmpPrefix, timeoutMs, clientInfo, scenario }) {
const command = process.argv[2];
const args = process.argv.slice(3);
if (!command) {
throw new Error(usage);
}
const client = new McpSmokeClient({ command, args, tmpPrefix });
const timeout = setTimeout(async () => {
await client.stop();
console.error(`MCP smoke timed out. stderr: ${client.stderrTail(2000)}`);
process.exit(1);
}, timeoutMs);
try {
await client.initialize(clientInfo);
const result = await scenario(client);
clearTimeout(timeout);
await client.stop();
console.log(JSON.stringify(result, null, 2));
} catch (error) {
clearTimeout(timeout);
await client.stop();
console.error(error.message);
const stderr = client.stderrTail(4000);
if (stderr) console.error(stderr);
process.exit(1);
}
}
class McpSmokeClient {
constructor({ command, args, tmpPrefix }) {
this.tmpDir = mkdtempSync(join(tmpdir(), tmpPrefix));
this.cidFile = join(this.tmpDir, "container.cid");
this.nextId = 1;
this.pending = new Map();
this.stdoutBuffer = "";
this.stderrBuffer = "";
this.exited = false;
this.child = spawn(command, args, {
cwd: new URL("..", import.meta.url).pathname,
env: { ...process.env, CONTEXT_KIT_DOCKER_CIDFILE: this.cidFile },
stdio: ["pipe", "pipe", "pipe"]
});
this.child.once("exit", (code, signal) => {
this.exited = true;
if (this.pending.size > 0) {
const error = new Error(`MCP child exited before responding (code=${code}, signal=${signal}). stderr: ${this.stderrTail(2000)}`);
for (const { reject } of this.pending.values()) reject(error);
this.pending.clear();
}
});
this.child.stderr.on("data", chunk => {
this.stderrBuffer += chunk.toString();
});
this.child.stdout.on("data", chunk => this.handleStdout(chunk));
}
stderrTail(length) {
return this.stderrBuffer.slice(-length);
}
handleStdout(chunk) {
this.stdoutBuffer += chunk.toString();
let newline;
while ((newline = this.stdoutBuffer.indexOf("\n")) >= 0) {
const line = this.stdoutBuffer.slice(0, newline).trim();
this.stdoutBuffer = this.stdoutBuffer.slice(newline + 1);
if (!line) continue;
let message;
try {
message = JSON.parse(line);
} catch {
continue;
}
if (message.id && this.pending.has(message.id)) {
const { resolve, reject } = this.pending.get(message.id);
this.pending.delete(message.id);
if (message.error) reject(new Error(JSON.stringify(message.error)));
else resolve(message.result);
}
}
}
request(method, params = {}) {
if (this.exited) {
return Promise.reject(new Error(`MCP child already exited. stderr: ${this.stderrTail(2000)}`));
}
const id = this.nextId++;
this.child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", id, method, params })}\n`);
return new Promise((resolve, reject) => this.pending.set(id, { resolve, reject }));
}
notify(method, params = {}) {
this.child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", method, params })}\n`);
}
async initialize(clientInfo) {
await this.request("initialize", {
protocolVersion: "2024-11-05",
capabilities: {},
clientInfo
});
this.notify("notifications/initialized");
}
async tools() {
const listed = await this.request("tools/list");
return new Set((listed.tools || []).map(tool => tool.name));
}
async requireTools(names) {
const toolNames = await this.tools();
for (const name of names) {
if (!toolNames.has(name)) throw new Error(`missing tool: ${name}`);
}
return toolNames;
}
callTool(name, args = {}) {
return this.request("tools/call", { name, arguments: args });
}
stopContainer() {
if (!existsSync(this.cidFile)) return;
const containerId = readFileSync(this.cidFile, "utf8").trim();
if (!containerId) return;
spawnSync("docker", ["stop", containerId], { stdio: "ignore" });
}
stop() {
return new Promise(resolve => {
if (this.exited) {
this.stopContainer();
rmSync(this.tmpDir, { recursive: true, force: true });
resolve();
return;
}
let stopTimer;
let termTimer;
let killTimer;
this.child.once("exit", () => {
this.stopContainer();
clearTimeout(stopTimer);
clearTimeout(termTimer);
clearTimeout(killTimer);
rmSync(this.tmpDir, { recursive: true, force: true });
resolve();
});
stopTimer = setTimeout(() => this.stopContainer(), 1000);
termTimer = setTimeout(() => {
if (!this.exited) this.child.kill("SIGTERM");
}, 3000);
killTimer = setTimeout(() => {
if (!this.exited) this.child.kill("SIGKILL");
}, 6000);
this.child.stdin.end();
});
}
}

57
scripts/release-check
View File

@@ -5,7 +5,29 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "${ROOT}" cd "${ROOT}"
tmp_dir="$(mktemp -d)" tmp_dir="$(mktemp -d)"
pick_port() {
python - <<'PY'
import socket
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.bind(("127.0.0.1", 0))
print(sock.getsockname()[1])
PY
}
release_id="release-$$"
export CONTEXT_KIT_COMPOSE_PROJECT="context-kit-${release_id}"
export CONTEXT_KIT_DATA_DIR="${tmp_dir}/data"
export CONTEXT_KIT_SEARXNG_PORT="$(pick_port)"
export CONTEXT_KIT_DOCS_PORT="$(pick_port)"
export CONTEXT_KIT_DOCS_SOURCES="config/sources.default.txt"
export CONTEXT_KIT_DOCS_LOCAL_SOURCES_DIR="${tmp_dir}/local-sources"
export CONTEXT_KIT_WEB_SEARCH_IMAGE="context-kit/web-search-mcp:${release_id}"
export CONTEXT_KIT_DOCS_IMAGE="context-kit/docs-mcp:${release_id}"
cleanup() { cleanup() {
docker compose -p "${CONTEXT_KIT_COMPOSE_PROJECT}" -f compose.yml down -v --remove-orphans >/dev/null 2>&1 || true
docker image rm "${CONTEXT_KIT_WEB_SEARCH_IMAGE}" "${CONTEXT_KIT_DOCS_IMAGE}" >/dev/null 2>&1 || true
rm -rf "${tmp_dir}" rm -rf "${tmp_dir}"
} }
trap cleanup EXIT trap cleanup EXIT
@@ -32,19 +54,47 @@ assert_redaction_check_does_not_disclose_matches() {
fi fi
} }
assert_web_search_image() {
docker run --rm --entrypoint node \
-e EXPECTED_MAX_BYTES="${CONTEXT_KIT_WEB_SEARCH_MAX_BYTES:-52428800}" \
"${CONTEXT_KIT_WEB_SEARCH_IMAGE}" \
-e '
const fs = require("node:fs");
const expected = Number(process.env.EXPECTED_MAX_BYTES) || 0;
const actual = Number(process.env.MAX_BYTES) || 0;
if (process.getuid && process.getuid() === 0) process.exit(1);
if (actual !== expected) process.exit(1);
const serverPath = "/usr/local/lib/node_modules/@zhafron/mcp-web-search/dist/src/server.js";
const server = fs.readFileSync(serverPath, "utf8");
if (!server.includes("max_download_bytes: z.number().int().min(1).max(MAX_BYTES).optional()")) process.exit(1);
const bingPath = "/usr/local/lib/node_modules/@zhafron/mcp-web-search/dist/src/providers/bing.js";
const bing = fs.readFileSync(bingPath, "utf8");
if (!bing.includes("Context Kit override for @zhafron/mcp-web-search 1.3.0")) process.exit(1);
if (!bing.includes("waitForSelector")) process.exit(1);
if (!bing.includes("decodeBingRedirect")) process.exit(1);
' >/dev/null
docker run --rm --entrypoint /usr/bin/test \
"${CONTEXT_KIT_WEB_SEARCH_IMAGE}" \
-x "${CONTEXT_KIT_WEB_SEARCH_CHROME_PATH:-/usr/bin/chromium}"
}
git diff --check HEAD git diff --check HEAD
git show --check --format= HEAD >/dev/null git show --check --format= HEAD >/dev/null
git ls-files --cached --error-unmatch \ git ls-files --cached --error-unmatch \
docker/web-search/patch-mcp-web-search.mjs \ docker/web-search/patch-mcp-web-search.mjs \
docker/web-search/overrides/bing.js \ docker/web-search/overrides/bing.js \
docker/docs/constraints.txt \ docker/docs/constraints.txt \
scripts/mcp-smoke-client.mjs \
scripts/smoke-web-search.mjs \ scripts/smoke-web-search.mjs \
scripts/smoke-docs.mjs \ scripts/smoke-docs.mjs \
scripts/release-check >/dev/null scripts/release-check >/dev/null
bash -n bin/context-kit bash -n bin/context-kit
bash -n scripts/release-check bash -n scripts/release-check
sh -n docker/docs/entrypoint.sh sh -n docker/docs/entrypoint.sh
check_node docker/web-search/patch-mcp-web-search.mjs docker/web-search/overrides/bing.js scripts/smoke-web-search.mjs scripts/smoke-docs.mjs check_node docker/web-search/patch-mcp-web-search.mjs docker/web-search/overrides/bing.js scripts/mcp-smoke-client.mjs scripts/smoke-web-search.mjs scripts/smoke-docs.mjs
node -e 'const fs=require("node:fs"); JSON.parse(fs.readFileSync("snippets/opencode.json", "utf8")); JSON.parse(fs.readFileSync("snippets/claude.mcp.json", "utf8"));' node -e 'const fs=require("node:fs"); JSON.parse(fs.readFileSync("snippets/opencode.json", "utf8")); JSON.parse(fs.readFileSync("snippets/claude.mcp.json", "utf8"));'
bin/context-kit install opencode > "${tmp_dir}/opencode.json" bin/context-kit install opencode > "${tmp_dir}/opencode.json"
@@ -60,13 +110,14 @@ bin/context-kit redaction-check "${tmp_dir}/opencode.json" "${tmp_dir}/claude.js
assert_redaction_check_does_not_disclose_matches assert_redaction_check_does_not_disclose_matches
bin/context-kit redaction-check bin/context-kit redaction-check
docker compose -p context-kit -f compose.yml config >/dev/null docker compose -p "${CONTEXT_KIT_COMPOSE_PROJECT}" -f compose.yml config >/dev/null
if env -u HOME docker compose --env-file /dev/null -p context-kit-release-home-check -f compose.yml config >"${tmp_dir}/compose-no-home.out" 2>"${tmp_dir}/compose-no-home.err"; then if env -u HOME -u CONTEXT_KIT_DATA_DIR -u CONTEXT_KIT_DOCS_LOCAL_SOURCES_DIR docker compose --env-file /dev/null -p context-kit-release-home-check -f compose.yml config >"${tmp_dir}/compose-no-home.out" 2>"${tmp_dir}/compose-no-home.err"; then
printf 'compose config unexpectedly succeeded without HOME or CONTEXT_KIT_DATA_DIR\n' >&2 printf 'compose config unexpectedly succeeded without HOME or CONTEXT_KIT_DATA_DIR\n' >&2
exit 1 exit 1
fi fi
CONTEXT_KIT_DATA_DIR="${tmp_dir}/compose-data" env -u HOME docker compose --env-file /dev/null -p context-kit-release-home-check -f compose.yml config >/dev/null CONTEXT_KIT_DATA_DIR="${tmp_dir}/compose-data" env -u HOME docker compose --env-file /dev/null -p context-kit-release-home-check -f compose.yml config >/dev/null
bin/context-kit build bin/context-kit build
assert_web_search_image
bin/context-kit restart bin/context-kit restart
bin/context-kit doctor bin/context-kit doctor
node scripts/smoke-web-search.mjs bin/context-kit web-search node scripts/smoke-web-search.mjs bin/context-kit web-search

194
scripts/smoke-docs.mjs
View File

@@ -1,170 +1,42 @@
import { spawn, spawnSync } from "node:child_process"; import { requireToolSuccess, runSmoke } from "./mcp-smoke-client.mjs";
import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
const command = process.argv[2]; const live = process.env.CONTEXT_KIT_LIVE_CHECKS === "1";
const args = process.argv.slice(3);
if (!command) { runSmoke({
throw new Error("usage: node scripts/smoke-docs.mjs <command> [args...]"); usage: "usage: node scripts/smoke-docs.mjs <command> [args...]",
} tmpPrefix: "context-kit-docs-smoke-",
timeoutMs: 300000,
clientInfo: { name: "context-kit-docs-smoke", version: "0.0.0" },
scenario: async client => {
const toolNames = await client.requireTools(["docs_query", "docs_sources"]);
const tmpDir = mkdtempSync(join(tmpdir(), "context-kit-docs-smoke-")); const sources = requireToolSuccess("docs_sources", await client.callTool("docs_sources"));
const cidFile = join(tmpDir, "container.cid"); if (!Array.isArray(sources?.structuredContent?.result)) {
const sourcesText = JSON.stringify(sources);
const child = spawn(command, args, { throw new Error(`docs_sources returned unexpected payload: ${sourcesText.slice(0, 500)}`);
cwd: new URL("..", import.meta.url).pathname,
env: { ...process.env, CONTEXT_KIT_DOCKER_CIDFILE: cidFile },
stdio: ["pipe", "pipe", "pipe"]
});
let nextId = 1;
const pending = new Map();
let stdoutBuffer = "";
let stderrBuffer = "";
let childExited = false;
child.once("exit", (code, signal) => {
childExited = true;
if (pending.size > 0) {
const error = new Error(`MCP child exited before responding (code=${code}, signal=${signal}). stderr: ${stderrBuffer.slice(-2000)}`);
for (const { reject } of pending.values()) reject(error);
pending.clear();
}
});
function stopChild() {
return new Promise(resolve => {
if (childExited) {
stopContainer();
rmSync(tmpDir, { recursive: true, force: true });
resolve();
return;
} }
child.stdin.end(); const result = {
const stopTimer = setTimeout(() => { tools: Array.from(toolNames).sort(),
stopContainer(); docs_sources: "pass"
}, 1000); };
const termTimer = setTimeout(() => {
if (!childExited) child.kill("SIGTERM");
}, 3000);
const killTimer = setTimeout(() => {
if (!childExited) child.kill("SIGKILL");
}, 6000);
child.once("exit", () => { if (live) {
stopContainer(); const query = requireToolSuccess("docs_query", await client.callTool("docs_query", {
clearTimeout(stopTimer); query: "Model Context Protocol documentation",
clearTimeout(termTimer); limit: 3,
clearTimeout(killTimer); auto_retrieve: true,
rmSync(tmpDir, { recursive: true, force: true }); auto_retrieve_threshold: 0.1,
resolve(); auto_retrieve_limit: 1,
}); max_bytes: 12000
}); }));
} const queryText = JSON.stringify(query);
if (!queryText.includes("search_results") && !queryText.includes("Model Context Protocol")) {
function stopContainer() { throw new Error(`docs_query returned unexpected payload: ${queryText.slice(0, 500)}`);
if (!existsSync(cidFile)) return; }
const containerId = readFileSync(cidFile, "utf8").trim(); result.docs_query = "pass";
if (!containerId) return;
spawnSync("docker", ["stop", containerId], { stdio: "ignore" });
}
const timeout = setTimeout(async () => {
await stopChild();
console.error(`Docs MCP smoke timed out. stderr: ${stderrBuffer.slice(-2000)}`);
process.exit(1);
}, 300000);
child.stderr.on("data", chunk => {
stderrBuffer += chunk.toString();
});
child.stdout.on("data", chunk => {
stdoutBuffer += chunk.toString();
let newline;
while ((newline = stdoutBuffer.indexOf("\n")) >= 0) {
const line = stdoutBuffer.slice(0, newline).trim();
stdoutBuffer = stdoutBuffer.slice(newline + 1);
if (!line) continue;
let message;
try {
message = JSON.parse(line);
} catch {
continue;
}
if (message.id && pending.has(message.id)) {
const { resolve, reject } = pending.get(message.id);
pending.delete(message.id);
if (message.error) reject(new Error(JSON.stringify(message.error)));
else resolve(message.result);
} }
return result;
} }
}); });
function request(method, params = {}) {
if (childExited) {
return Promise.reject(new Error(`MCP child already exited. stderr: ${stderrBuffer.slice(-2000)}`));
}
const id = nextId++;
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", id, method, params })}\n`);
return new Promise((resolve, reject) => pending.set(id, { resolve, reject }));
}
function notify(method, params = {}) {
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", method, params })}\n`);
}
async function callTool(name, args = {}) {
return request("tools/call", { name, arguments: args });
}
try {
await request("initialize", {
protocolVersion: "2024-11-05",
capabilities: {},
clientInfo: { name: "context-kit-docs-smoke", version: "0.0.0" }
});
notify("notifications/initialized");
const listed = await request("tools/list");
const toolNames = new Set((listed.tools || []).map(tool => tool.name));
for (const name of ["docs_query", "docs_sources"]) {
if (!toolNames.has(name)) throw new Error(`missing tool: ${name}`);
}
const sources = await callTool("docs_sources");
const sourcesText = JSON.stringify(sources);
if (sources.isError) {
throw new Error(`docs_sources returned an error: ${sourcesText.slice(0, 500)}`);
}
const query = await callTool("docs_query", {
query: "Model Context Protocol documentation",
limit: 3,
auto_retrieve: true,
auto_retrieve_threshold: 0.1,
auto_retrieve_limit: 1,
max_bytes: 12000
});
const queryText = JSON.stringify(query);
if (!queryText.includes("search_results") && !queryText.includes("Model Context Protocol")) {
throw new Error(`docs_query returned unexpected payload: ${queryText.slice(0, 500)}`);
}
clearTimeout(timeout);
await stopChild();
console.log(JSON.stringify({
tools: Array.from(toolNames).sort(),
docs_sources: "pass",
docs_query: "pass"
}, null, 2));
} catch (error) {
clearTimeout(timeout);
await stopChild();
console.error(error.message);
if (stderrBuffer) console.error(stderrBuffer.slice(-4000));
process.exit(1);
}

238
scripts/smoke-web-search.mjs
View File

@@ -1,197 +1,63 @@
import { spawn, spawnSync } from "node:child_process"; import { requireToolSuccess, runSmoke, textFrom } from "./mcp-smoke-client.mjs";
import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
const command = process.argv[2]; const live = process.env.CONTEXT_KIT_LIVE_CHECKS === "1";
const args = process.argv.slice(3);
if (!command) { runSmoke({
throw new Error("usage: node scripts/smoke-web-search.mjs <command> [args...]"); usage: "usage: node scripts/smoke-web-search.mjs <command> [args...]",
} tmpPrefix: "context-kit-web-search-smoke-",
timeoutMs: 120000,
clientInfo: { name: "context-kit-web-search-smoke", version: "0.0.0" },
scenario: async client => {
const toolNames = await client.requireTools(["search_web", "fetch_url"]);
const tmpDir = mkdtempSync(join(tmpdir(), "context-kit-web-search-smoke-")); const localResult = await client.callTool("fetch_url", {
const cidFile = join(tmpDir, "container.cid"); url: "http://127.0.0.1:1/",
max_download_bytes: 52428800
const child = spawn(command, args, {
cwd: new URL("..", import.meta.url).pathname,
env: { ...process.env, CONTEXT_KIT_DOCKER_CIDFILE: cidFile },
stdio: ["pipe", "pipe", "pipe"]
});
let nextId = 1;
const pending = new Map();
let stdoutBuffer = "";
let stderrBuffer = "";
let childExited = false;
child.once("exit", (code, signal) => {
childExited = true;
if (pending.size > 0) {
const error = new Error(`MCP child exited before responding (code=${code}, signal=${signal}). stderr: ${stderrBuffer.slice(-2000)}`);
for (const { reject } of pending.values()) reject(error);
pending.clear();
}
});
function stopChild() {
return new Promise(resolve => {
if (childExited) {
stopContainer();
rmSync(tmpDir, { recursive: true, force: true });
resolve();
return;
}
child.stdin.end();
const stopTimer = setTimeout(() => {
stopContainer();
}, 1000);
const termTimer = setTimeout(() => {
if (!childExited) child.kill("SIGTERM");
}, 3000);
const killTimer = setTimeout(() => {
if (!childExited) child.kill("SIGKILL");
}, 6000);
child.once("exit", () => {
stopContainer();
clearTimeout(stopTimer);
clearTimeout(termTimer);
clearTimeout(killTimer);
rmSync(tmpDir, { recursive: true, force: true });
resolve();
}); });
}); const localBlocked = Boolean(localResult.isError) && textFrom(localResult).includes("Blocked localhost/private URL");
} if (!localBlocked) throw new Error("localhost/private URL was not blocked as expected");
function stopContainer() { const result = {
if (!existsSync(cidFile)) return; tools: Array.from(toolNames).sort(),
const containerId = readFileSync(cidFile, "utf8").trim(); localhost_guard: "pass"
if (!containerId) return; };
spawnSync("docker", ["stop", containerId], { stdio: "ignore" });
}
const timeout = setTimeout(async () => { if (live) {
await stopChild(); const searxng = textFrom(requireToolSuccess("search_web/searxng", await client.callTool("search_web", {
console.error(`MCP smoke timed out. stderr: ${stderrBuffer.slice(-2000)}`); q: "Model Context Protocol",
process.exit(1); limit: 2,
}, 120000); provider: "searxng"
})));
if (!searxng.includes("Model")) throw new Error(`SearXNG smoke returned unexpected text: ${searxng.slice(0, 500)}`);
child.stderr.on("data", chunk => { const bing = textFrom(requireToolSuccess("search_web/bing", await client.callTool("search_web", {
stderrBuffer += chunk.toString(); q: "Model Context Protocol",
}); limit: 2,
provider: "bing"
})));
if (!bing.includes("Model")) throw new Error(`Bing smoke returned unexpected text: ${bing.slice(0, 500)}`);
child.stdout.on("data", chunk => { const fetch = textFrom(requireToolSuccess("fetch_url/http", await client.callTool("fetch_url", {
stdoutBuffer += chunk.toString(); url: "https://example.com/",
let newline; format: "markdown",
while ((newline = stdoutBuffer.indexOf("\n")) >= 0) { max_download_bytes: 52428800
const line = stdoutBuffer.slice(0, newline).trim(); })));
stdoutBuffer = stdoutBuffer.slice(newline + 1); if (!fetch.includes("Example Domain")) throw new Error(`fetch smoke returned unexpected text: ${fetch.slice(0, 500)}`);
if (!line) continue;
let message; const browserFetch = textFrom(requireToolSuccess("fetch_url/browser", await client.callTool("fetch_url", {
try { url: "https://example.com/",
message = JSON.parse(line); format: "markdown",
} catch { engine: "browser",
continue; max_download_bytes: 52428800
} })));
if (message.id && pending.has(message.id)) { if (!browserFetch.includes("Example Domain")) throw new Error(`browser fetch smoke returned unexpected text: ${browserFetch.slice(0, 500)}`);
const { resolve, reject } = pending.get(message.id);
pending.delete(message.id); result.searxng = "pass";
if (message.error) reject(new Error(JSON.stringify(message.error))); result.bing = "pass";
else resolve(message.result); result.fetch_url = "pass";
result.fetch_url_browser_engine_currently_http = "pass";
} }
return result;
} }
}); });
function request(method, params = {}) {
if (childExited) {
return Promise.reject(new Error(`MCP child already exited. stderr: ${stderrBuffer.slice(-2000)}`));
}
const id = nextId++;
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", id, method, params })}\n`);
return new Promise((resolve, reject) => pending.set(id, { resolve, reject }));
}
function notify(method, params = {}) {
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", method, params })}\n`);
}
function textFrom(result) {
return (result.content || [])
.filter(part => part.type === "text")
.map(part => part.text)
.join("\n");
}
async function callTool(name, args = {}) {
return request("tools/call", { name, arguments: args });
}
try {
await request("initialize", {
protocolVersion: "2024-11-05",
capabilities: {},
clientInfo: { name: "context-kit-smoke", version: "0.0.0" }
});
notify("notifications/initialized");
const listed = await request("tools/list");
const toolNames = new Set((listed.tools || []).map(tool => tool.name));
for (const name of ["search_web", "fetch_url"]) {
if (!toolNames.has(name)) throw new Error(`missing tool: ${name}`);
}
const searxng = textFrom(await callTool("search_web", {
q: "Model Context Protocol",
limit: 2,
provider: "searxng"
}));
if (!searxng.includes("Model")) throw new Error(`SearXNG smoke returned unexpected text: ${searxng.slice(0, 500)}`);
const bing = textFrom(await callTool("search_web", {
q: "Model Context Protocol",
limit: 2,
provider: "bing"
}));
if (!bing.includes("Model")) throw new Error(`Bing smoke returned unexpected text: ${bing.slice(0, 500)}`);
const fetch = textFrom(await callTool("fetch_url", {
url: "https://example.com/",
format: "markdown",
max_download_bytes: 52428800
}));
if (!fetch.includes("Example Domain")) throw new Error(`fetch smoke returned unexpected text: ${fetch.slice(0, 500)}`);
const browserFetch = textFrom(await callTool("fetch_url", {
url: "https://example.com/",
format: "markdown",
engine: "browser",
max_download_bytes: 52428800
}));
if (!browserFetch.includes("Example Domain")) throw new Error(`browser fetch smoke returned unexpected text: ${browserFetch.slice(0, 500)}`);
const localResult = await callTool("fetch_url", {
url: "http://127.0.0.1:1/",
max_download_bytes: 52428800
});
const localBlocked = Boolean(localResult.isError) && textFrom(localResult).includes("Blocked localhost/private URL");
if (!localBlocked) throw new Error("localhost/private URL was not blocked as expected");
clearTimeout(timeout);
await stopChild();
console.log(JSON.stringify({
tools: Array.from(toolNames).sort(),
searxng: "pass",
bing: "pass",
fetch_url: "pass",
fetch_url_browser_engine_currently_http: "pass",
localhost_guard: "pass"
}, null, 2));
} catch (error) {
clearTimeout(timeout);
await stopChild();
console.error(error.message);
if (stderrBuffer) console.error(stderrBuffer.slice(-4000));
process.exit(1);
}