Simplify runtime checks and MCP smokes
This commit is contained in:
42
README.md
42
README.md
@@ -6,13 +6,13 @@ Local web search. Local docs. Repo packing. No API keys required.
|
||||
|
||||
## What You Get
|
||||
|
||||
Context Kit gives coding agents three local MCP servers:
|
||||
Context Kit gives coding agents three local tools:
|
||||
|
||||
| Server | Purpose | Default |
|
||||
|---|---|---|
|
||||
| `context-web-search` | Current web search through local SearXNG plus URL fetch/extract | Enabled |
|
||||
| `context-docs` | Semantic search over curated `llms.txt` documentation | Enabled |
|
||||
| `context-repomix` | Pack local or remote repositories into AI-friendly context | Enabled |
|
||||
| Tool | Purpose |
|
||||
|---|---|
|
||||
| `context-web-search` | Current web search through local SearXNG plus URL fetch/extract |
|
||||
| `context-docs` | Semantic search over curated `llms.txt` documentation |
|
||||
| `context-repomix` | Pack repositories into AI-friendly context |
|
||||
|
||||
The first public release deliberately keeps the surface area small: web search,
|
||||
docs search, and repository packing.
|
||||
@@ -53,23 +53,17 @@ The default snippet uses `context-kit` on `PATH`. Use
|
||||
`bin/context-kit install opencode --absolute` only for private, machine-local
|
||||
config that will not be committed.
|
||||
|
||||
## Defaults
|
||||
## How It Runs
|
||||
|
||||
- SearXNG binds to `127.0.0.1:8099` only.
|
||||
- `context-web-search` defaults `search_web` to SearXNG, then falls back to
|
||||
DuckDuckGo and Bing. Bing uses Chromium inside the web-search image.
|
||||
- `fetch_url` uses upstream HTTP extraction. In `mcp-web-search` 1.3.0,
|
||||
`engine=browser` is accepted but does not invoke Chromium yet.
|
||||
- `context-docs` runs as a long-lived service on `127.0.0.1:8776` (Streamable
|
||||
HTTP MCP) so every client shares one indexer and one Chroma writer. The
|
||||
`bin/context-kit docs` stdio command is kept as a compatibility shim for
|
||||
clients that cannot speak HTTP MCP.
|
||||
- `context-web-search` and `context-repomix` run as local stdio MCP commands.
|
||||
- `context-docs` runs as a local HTTP MCP service. `bin/context-kit docs` is a
|
||||
stdio fallback for clients that cannot use HTTP MCP.
|
||||
- `context-docs` browser CORS is disabled by default; set exact local origins
|
||||
only when a browser-based client needs direct access.
|
||||
- Docs and model caches live in `$HOME/.local/share/context-kit`.
|
||||
- Docs refresh TTL defaults to `24h`.
|
||||
- MCP containers are labeled `dev.context-kit=true` for safe inspection and cleanup.
|
||||
- Repomix mounts only the current project read-only, not your whole home directory.
|
||||
- Repomix mounts only the current project read-only.
|
||||
- No code-editing MCP server is enabled by default.
|
||||
|
||||
## Docs Sources
|
||||
@@ -91,9 +85,12 @@ Example:
|
||||
|
||||
```sh
|
||||
CONTEXT_KIT_DOCS_SOURCES="config/sources.default.txt config/sources.js.txt" \
|
||||
bin/context-kit docs
|
||||
bin/context-kit restart
|
||||
```
|
||||
|
||||
Source changes are loaded by `start`/`restart`; `bin/context-kit docs` is only a
|
||||
stdio bridge to the already-running docs service.
|
||||
|
||||
Large vendor feeds are opt-in because they can expand to thousands of sections
|
||||
and take a while to embed.
|
||||
|
||||
@@ -105,6 +102,8 @@ bin/context-kit stop
|
||||
bin/context-kit build
|
||||
bin/context-kit status
|
||||
bin/context-kit doctor
|
||||
bin/context-kit install claude
|
||||
bin/context-kit install opencode
|
||||
bin/context-kit redaction-check
|
||||
```
|
||||
|
||||
@@ -116,6 +115,13 @@ bin/context-kit docs
|
||||
bin/context-kit repomix
|
||||
```
|
||||
|
||||
After pulling Context Kit updates, rebuild local images and restart services:
|
||||
|
||||
```sh
|
||||
bin/context-kit build
|
||||
bin/context-kit restart
|
||||
```
|
||||
|
||||
## Security Model
|
||||
|
||||
Context Kit is local-first, but MCP tools still extend what your agent can do.
|
||||
|
||||
146
bin/context-kit
146
bin/context-kit
@@ -42,22 +42,26 @@ fail() {
|
||||
|
||||
load_env_file
|
||||
|
||||
DEFAULT_DATA_DIR="${HOME:-${PWD}}/.local/share/context-kit"
|
||||
if [[ -z "${CONTEXT_KIT_DATA_DIR:-}" && -z "${HOME:-}" ]]; then
|
||||
fail "HOME or CONTEXT_KIT_DATA_DIR must be set"
|
||||
fi
|
||||
|
||||
DEFAULT_DATA_DIR="${HOME:-}/.local/share/context-kit"
|
||||
PROJECT="${CONTEXT_KIT_COMPOSE_PROJECT:-context-kit}"
|
||||
COMPOSE_FILE="${ROOT}/compose.yml"
|
||||
DATA_DIR="${CONTEXT_KIT_DATA_DIR:-${DEFAULT_DATA_DIR}}"
|
||||
NETWORK="${CONTEXT_KIT_DOCKER_NETWORK:-${PROJECT}_default}"
|
||||
NETWORK="${PROJECT}_default"
|
||||
SEARXNG_PORT="${CONTEXT_KIT_SEARXNG_PORT:-8099}"
|
||||
DOCS_PORT="${CONTEXT_KIT_DOCS_PORT:-8776}"
|
||||
DOCS_HTTP_URL="${CONTEXT_KIT_DOCS_HTTP_URL:-http://127.0.0.1:${DOCS_PORT}/mcp}"
|
||||
WEB_SEARCH_MAX_BYTES="${CONTEXT_KIT_WEB_SEARCH_MAX_BYTES:-52428800}"
|
||||
WEB_SEARCH_PROVIDER="${CONTEXT_KIT_WEB_SEARCH_PROVIDER:-${DEFAULT_SEARCH_PROVIDER:-searxng}}"
|
||||
WEB_SEARCH_HTTP_TIMEOUT="${CONTEXT_KIT_WEB_SEARCH_HTTP_TIMEOUT:-${HTTP_TIMEOUT:-15000}}"
|
||||
WEB_SEARCH_MAX_RESULTS="${CONTEXT_KIT_WEB_SEARCH_MAX_RESULTS:-${MAX_RESULTS:-10}}"
|
||||
WEB_SEARCH_CHROME_PATH="${CONTEXT_KIT_WEB_SEARCH_CHROME_PATH:-${CHROME_PATH:-/usr/bin/chromium}}"
|
||||
WEB_SEARCH_BROWSER_USER_AGENT="${CONTEXT_KIT_WEB_SEARCH_BROWSER_USER_AGENT:-${BROWSER_SEARCH_USER_AGENT:-}}"
|
||||
WEB_SEARCH_MCP_COMPAT_MODE="${CONTEXT_KIT_WEB_SEARCH_MCP_COMPAT_MODE:-${MCP_COMPAT_MODE:-}}"
|
||||
DOCS_CONTAINER_NAME="context-kit-docs-mcp"
|
||||
WEB_SEARCH_PROVIDER="${CONTEXT_KIT_WEB_SEARCH_PROVIDER:-searxng}"
|
||||
WEB_SEARCH_HTTP_TIMEOUT="${CONTEXT_KIT_WEB_SEARCH_HTTP_TIMEOUT:-15000}"
|
||||
WEB_SEARCH_MAX_RESULTS="${CONTEXT_KIT_WEB_SEARCH_MAX_RESULTS:-10}"
|
||||
WEB_SEARCH_CHROME_PATH="${CONTEXT_KIT_WEB_SEARCH_CHROME_PATH:-/usr/bin/chromium}"
|
||||
WEB_SEARCH_BROWSER_USER_AGENT="${CONTEXT_KIT_WEB_SEARCH_BROWSER_USER_AGENT:-}"
|
||||
WEB_SEARCH_MCP_COMPAT_MODE="${CONTEXT_KIT_WEB_SEARCH_MCP_COMPAT_MODE:-}"
|
||||
DOCS_SERVICE_NAME="docs-mcp"
|
||||
DOCS_SOURCES_FILE="${DATA_DIR}/docs-sources.txt"
|
||||
DOCS_DATA_DIR="${DATA_DIR}/docs"
|
||||
MODELS_DATA_DIR="${DATA_DIR}/models"
|
||||
@@ -108,10 +112,18 @@ compose() {
|
||||
CONTEXT_KIT_DOCS_PREINDEX="${CONTEXT_KIT_DOCS_PREINDEX:-0}" \
|
||||
CONTEXT_KIT_DOCS_LOCAL_SOURCES_DIR="${DOCS_LOCAL_SOURCES_DIR}" \
|
||||
CONTEXT_KIT_DOCS_LOCAL_SOURCES_PORT="${DOCS_LOCAL_SOURCES_PORT}" \
|
||||
BUILDX_BUILDER="${CONTEXT_KIT_BUILDX_BUILDER:-${BUILDX_BUILDER:-default}}" \
|
||||
CONTEXT_KIT_WEB_SEARCH_IMAGE="${WEB_SEARCH_IMAGE}" \
|
||||
CONTEXT_KIT_DOCS_IMAGE="${DOCS_IMAGE}" \
|
||||
BUILDX_BUILDER="${CONTEXT_KIT_BUILDX_BUILDER:-default}" \
|
||||
docker compose -p "${PROJECT}" -f "${COMPOSE_FILE}" "$@"
|
||||
}
|
||||
|
||||
require_no_args() {
|
||||
local usage_text="$1"
|
||||
shift
|
||||
[[ "$#" -eq 0 ]] || fail "${usage_text}"
|
||||
}
|
||||
|
||||
write_docs_sources_file() {
|
||||
mkdir -p "$(dirname "${DOCS_SOURCES_FILE}")"
|
||||
local tmp="${DOCS_SOURCES_FILE}.tmp.$$"
|
||||
@@ -152,41 +164,6 @@ check_data_dirs() {
|
||||
return "${ok}"
|
||||
}
|
||||
|
||||
check_web_search_schema_patch() {
|
||||
docker run --rm --entrypoint node \
|
||||
-e MAX_BYTES="${WEB_SEARCH_MAX_BYTES}" \
|
||||
-e EXPECTED_MAX_BYTES="${WEB_SEARCH_MAX_BYTES}" \
|
||||
"${WEB_SEARCH_IMAGE}" \
|
||||
-e '
|
||||
const fs = require("node:fs");
|
||||
const expected = Number(process.env.EXPECTED_MAX_BYTES) || 0;
|
||||
const actual = Number(process.env.MAX_BYTES) || 0;
|
||||
const serverPath = "/usr/local/lib/node_modules/@zhafron/mcp-web-search/dist/src/server.js";
|
||||
const source = fs.readFileSync(serverPath, "utf8");
|
||||
if (actual !== expected) process.exit(1);
|
||||
if (!source.includes("max_download_bytes: z.number().int().min(1).max(MAX_BYTES).optional()")) process.exit(1);
|
||||
' >/dev/null 2>&1
|
||||
}
|
||||
|
||||
check_web_search_bing_override() {
|
||||
docker run --rm --entrypoint node \
|
||||
"${WEB_SEARCH_IMAGE}" \
|
||||
-e '
|
||||
const fs = require("node:fs");
|
||||
const bingPath = "/usr/local/lib/node_modules/@zhafron/mcp-web-search/dist/src/providers/bing.js";
|
||||
const source = fs.readFileSync(bingPath, "utf8");
|
||||
if (!source.includes("Context Kit override for @zhafron/mcp-web-search 1.3.0")) process.exit(1);
|
||||
if (!source.includes("waitForSelector")) process.exit(1);
|
||||
if (!source.includes("decodeBingRedirect")) process.exit(1);
|
||||
' >/dev/null 2>&1
|
||||
}
|
||||
|
||||
check_web_search_chrome() {
|
||||
docker run --rm --entrypoint /usr/bin/test \
|
||||
"${WEB_SEARCH_IMAGE}" \
|
||||
-x "${WEB_SEARCH_CHROME_PATH}" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
warn() {
|
||||
printf 'warn: %s\n' "$*" >&2
|
||||
}
|
||||
@@ -239,21 +216,35 @@ wait_for_searxng() {
|
||||
done
|
||||
|
||||
warn "SearXNG did not become ready on 127.0.0.1:${SEARXNG_PORT} after 30s"
|
||||
return 1
|
||||
}
|
||||
|
||||
docs_service_running() {
|
||||
local container_id
|
||||
container_id="$(compose ps -q "${DOCS_SERVICE_NAME}" 2>/dev/null || true)"
|
||||
[[ -n "${container_id}" ]] || return 1
|
||||
docker inspect -f '{{.State.Running}}' "${container_id}" 2>/dev/null | grep -qx true
|
||||
}
|
||||
|
||||
wait_for_docs_mcp() {
|
||||
command -v curl >/dev/null 2>&1 || return 0
|
||||
|
||||
# First-run can take a while: model download + full preindex of every source.
|
||||
local attempt
|
||||
# First run can take a while: model download plus optional eager preindexing.
|
||||
local attempt http_ready=0
|
||||
for attempt in {1..180}; do
|
||||
if curl -fsS -o /dev/null "http://127.0.0.1:${DOCS_PORT}/status" 2>/dev/null; then
|
||||
return 0
|
||||
http_ready=1
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
|
||||
warn "docs-mcp did not become ready on 127.0.0.1:${DOCS_PORT} after 180s (check: docker logs ${DOCS_CONTAINER_NAME})"
|
||||
if [[ "${http_ready}" -ne 1 ]]; then
|
||||
warn "docs-mcp did not become ready on 127.0.0.1:${DOCS_PORT} after 180s (check: docker compose logs ${DOCS_SERVICE_NAME})"
|
||||
return 1
|
||||
fi
|
||||
|
||||
return 0
|
||||
}
|
||||
|
||||
abs_dir() {
|
||||
@@ -293,7 +284,7 @@ resolved_sources() {
|
||||
}
|
||||
|
||||
cmd_build() {
|
||||
[[ "$#" -eq 0 ]] || fail "usage: context-kit build"
|
||||
require_no_args "usage: context-kit build" "$@"
|
||||
require_docker
|
||||
# web-search-mcp is still profile-gated (built but not auto-started);
|
||||
# docs-mcp is a regular long-lived service so it builds without a profile.
|
||||
@@ -303,6 +294,7 @@ cmd_build() {
|
||||
}
|
||||
|
||||
cmd_start() {
|
||||
require_no_args "usage: context-kit start" "$@"
|
||||
require_docker
|
||||
prepare_data_dirs
|
||||
if ! docker image inspect "${WEB_SEARCH_IMAGE}" >/dev/null 2>&1 || ! docker image inspect "${DOCS_IMAGE}" >/dev/null 2>&1; then
|
||||
@@ -315,11 +307,19 @@ cmd_start() {
|
||||
}
|
||||
|
||||
cmd_stop() {
|
||||
require_no_args "usage: context-kit stop" "$@"
|
||||
require_docker
|
||||
compose stop searxng docs-mcp
|
||||
}
|
||||
|
||||
cmd_restart() {
|
||||
require_no_args "usage: context-kit restart" "$@"
|
||||
cmd_stop
|
||||
cmd_start
|
||||
}
|
||||
|
||||
cmd_status() {
|
||||
require_no_args "usage: context-kit status" "$@"
|
||||
require_docker
|
||||
printf 'Services\n'
|
||||
compose ps
|
||||
@@ -327,9 +327,9 @@ cmd_status() {
|
||||
docker image ls --format '{{.Repository}}:{{.Tag}}\t{{.Size}}' \
|
||||
| grep -E '^(context-kit/|ghcr.io/yamadashy/repomix:)' || true
|
||||
printf '\nActive per-call MCP containers\n'
|
||||
docker ps -a --filter label=dev.context-kit=true --format '{{.Names}}\t{{.Status}}\t{{.Image}}\t{{.Command}}' \
|
||||
| awk 'BEGIN { print "NAMES\tSTATUS\tIMAGE\tCOMMAND" } $1 !~ /^context-kit-(docs-mcp|searxng-1)$/ { print }'
|
||||
printf '\nDocs MCP endpoint\n- %s (container: %s)\n' "${DOCS_HTTP_URL}" "${DOCS_CONTAINER_NAME}"
|
||||
docker ps -a --filter label=dev.context-kit=true --format '{{.Names}}\t{{.Status}}\t{{.Image}}\t{{.Command}}\t{{.Label "com.docker.compose.service"}}' \
|
||||
| awk -F '\t' 'BEGIN { print "NAMES\tSTATUS\tIMAGE\tCOMMAND" } $5 !~ /^(searxng|docs-mcp)$/ { print $1 "\t" $2 "\t" $3 "\t" $4 }'
|
||||
printf '\nDocs MCP endpoint\n- %s (service: %s)\n' "${DOCS_HTTP_URL}" "${DOCS_SERVICE_NAME}"
|
||||
printf '\nDocs sources\n'
|
||||
resolved_sources | sed 's/^/- /'
|
||||
printf '\nLocal docs source directory\n- %s (served inside docs-mcp at http://127.0.0.1:%s/)\n' "${DOCS_LOCAL_SOURCES_DIR}" "${DOCS_LOCAL_SOURCES_PORT}"
|
||||
@@ -337,6 +337,7 @@ cmd_status() {
|
||||
}
|
||||
|
||||
cmd_doctor() {
|
||||
require_no_args "usage: context-kit doctor" "$@"
|
||||
local ok=0
|
||||
printf 'Context Kit doctor\n'
|
||||
|
||||
@@ -376,27 +377,6 @@ cmd_doctor() {
|
||||
fi
|
||||
done
|
||||
|
||||
if docker image inspect "${WEB_SEARCH_IMAGE}" >/dev/null 2>&1; then
|
||||
if check_web_search_schema_patch; then
|
||||
printf 'pass web-search fetch_url max-bytes schema patch: %s\n' "${WEB_SEARCH_MAX_BYTES}"
|
||||
else
|
||||
printf 'fail web-search max-bytes schema patch missing; run: context-kit build\n'
|
||||
ok=1
|
||||
fi
|
||||
if check_web_search_bing_override; then
|
||||
printf 'pass web-search Bing provider override installed\n'
|
||||
else
|
||||
printf 'fail web-search Bing provider override missing; run: context-kit build\n'
|
||||
ok=1
|
||||
fi
|
||||
if check_web_search_chrome; then
|
||||
printf 'pass web-search Chromium path: %s\n' "${WEB_SEARCH_CHROME_PATH}"
|
||||
else
|
||||
printf 'fail web-search Chromium path unavailable: %s\n' "${WEB_SEARCH_CHROME_PATH}"
|
||||
ok=1
|
||||
fi
|
||||
fi
|
||||
|
||||
if command -v curl >/dev/null 2>&1 && curl -fsS "http://127.0.0.1:${SEARXNG_PORT}/healthz" >/dev/null 2>&1; then
|
||||
printf 'pass SearXNG responds on 127.0.0.1:%s\n' "${SEARXNG_PORT}"
|
||||
else
|
||||
@@ -421,6 +401,7 @@ cmd_doctor() {
|
||||
}
|
||||
|
||||
cmd_web_search() {
|
||||
require_no_args "usage: context-kit web-search" "$@"
|
||||
require_docker
|
||||
require_network
|
||||
require_image "${WEB_SEARCH_IMAGE}" "context-kit build"
|
||||
@@ -433,7 +414,7 @@ cmd_web_search() {
|
||||
"${cidfile_args[@]}" \
|
||||
--network "${NETWORK}" \
|
||||
-e DEFAULT_SEARCH_PROVIDER="${WEB_SEARCH_PROVIDER}" \
|
||||
-e SEARXNG_URL="${SEARXNG_URL:-http://searxng:8080}" \
|
||||
-e SEARXNG_URL="http://searxng:8080" \
|
||||
-e CHROME_PATH="${WEB_SEARCH_CHROME_PATH}" \
|
||||
-e HTTP_TIMEOUT="${WEB_SEARCH_HTTP_TIMEOUT}" \
|
||||
-e MAX_BYTES="${WEB_SEARCH_MAX_BYTES}" \
|
||||
@@ -444,6 +425,7 @@ cmd_web_search() {
|
||||
}
|
||||
|
||||
cmd_docs() {
|
||||
require_no_args "usage: context-kit docs" "$@"
|
||||
# Prefer the `type: remote` MCP config pointing at ${DOCS_HTTP_URL}.
|
||||
# This stdio entrypoint is kept for clients that cannot speak HTTP MCP:
|
||||
# it spawns a thin mcp-proxy bridge per call but all calls multiplex onto
|
||||
@@ -453,11 +435,11 @@ cmd_docs() {
|
||||
require_network
|
||||
require_image "${DOCS_IMAGE}" "context-kit build"
|
||||
|
||||
if ! docker ps --filter "name=^${DOCS_CONTAINER_NAME}$" --filter "status=running" --format '{{.Names}}' | grep -qx "${DOCS_CONTAINER_NAME}"; then
|
||||
if ! docs_service_running; then
|
||||
fail "long-lived docs-mcp not running; start it with: context-kit start"
|
||||
fi
|
||||
|
||||
local bridge_url="http://${DOCS_CONTAINER_NAME}:8000/mcp"
|
||||
local bridge_url="http://${DOCS_SERVICE_NAME}:8000/mcp"
|
||||
local cidfile_args=()
|
||||
if [[ -n "${CONTEXT_KIT_DOCKER_CIDFILE:-}" ]]; then
|
||||
cidfile_args=(--cidfile "${CONTEXT_KIT_DOCKER_CIDFILE}")
|
||||
@@ -473,6 +455,7 @@ cmd_docs() {
|
||||
}
|
||||
|
||||
cmd_repomix() {
|
||||
require_no_args "usage: context-kit repomix" "$@"
|
||||
require_docker
|
||||
require_image "${REPOMIX_IMAGE}" "docker pull ${REPOMIX_IMAGE}"
|
||||
local dir mount_dir
|
||||
@@ -557,9 +540,12 @@ JSON
|
||||
cmd_install() {
|
||||
local target="${1:-}"
|
||||
shift || true
|
||||
local option="${1:-}"
|
||||
shift || true
|
||||
[[ "$#" -eq 0 ]] || fail "usage: context-kit install claude|opencode [--absolute]"
|
||||
case "${target}" in
|
||||
opencode) print_opencode "${1:-}" ;;
|
||||
claude) print_claude "${1:-}" ;;
|
||||
opencode) print_opencode "${option}" ;;
|
||||
claude) print_claude "${option}" ;;
|
||||
*) fail "usage: context-kit install claude|opencode [--absolute]" ;;
|
||||
esac
|
||||
}
|
||||
@@ -611,7 +597,7 @@ cmd_redaction_check() {
|
||||
case "${1:-}" in
|
||||
start) shift; cmd_start "$@" ;;
|
||||
stop) shift; cmd_stop "$@" ;;
|
||||
restart) shift; cmd_stop; cmd_start "$@" ;;
|
||||
restart) shift; cmd_restart "$@" ;;
|
||||
build) shift; cmd_build "$@" ;;
|
||||
status) shift; cmd_status "$@" ;;
|
||||
doctor) shift; cmd_doctor "$@" ;;
|
||||
|
||||
@@ -20,7 +20,7 @@ services:
|
||||
context: ./docker/web-search
|
||||
args:
|
||||
MCP_WEB_SEARCH_MAX_BYTES: "${CONTEXT_KIT_WEB_SEARCH_MAX_BYTES:-52428800}"
|
||||
image: context-kit/web-search-mcp:latest
|
||||
image: ${CONTEXT_KIT_WEB_SEARCH_IMAGE:-context-kit/web-search-mcp:latest}
|
||||
profiles: ["mcp"]
|
||||
stdin_open: true
|
||||
tty: false
|
||||
@@ -39,10 +39,9 @@ services:
|
||||
docs-mcp:
|
||||
build:
|
||||
context: ./docker/docs
|
||||
image: context-kit/docs-mcp:latest
|
||||
image: ${CONTEXT_KIT_DOCS_IMAGE:-context-kit/docs-mcp:latest}
|
||||
# Long-lived shared docs MCP. One container = one Chroma writer; clients
|
||||
# connect over Streamable HTTP (mcp-proxy bridges llms-txt-mcp's stdio).
|
||||
container_name: context-kit-docs-mcp
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "127.0.0.1:${CONTEXT_KIT_DOCS_PORT:-8776}:8000"
|
||||
|
||||
@@ -15,6 +15,12 @@ sources_file="${DOCS_MCP_SOURCES_FILE:-/etc/context-kit/docs-sources.txt}"
|
||||
local_sources_dir="${DOCS_MCP_LOCAL_SOURCES_DIR:-/etc/context-kit/local-sources}"
|
||||
local_sources_port="${DOCS_MCP_LOCAL_SOURCES_PORT:-8769}"
|
||||
|
||||
if [ ! -f "$sources_file" ]; then
|
||||
echo "docs-mcp: sources file missing: $sources_file" >&2
|
||||
echo "docs-mcp: run bin/context-kit start to generate it, or mount a file at that path." >&2
|
||||
exit 64
|
||||
fi
|
||||
|
||||
if [ ! -r "$sources_file" ]; then
|
||||
echo "docs-mcp: sources file not readable: $sources_file" >&2
|
||||
echo "docs-mcp: set DOCS_MCP_SOURCES_FILE or mount one at that path." >&2
|
||||
|
||||
@@ -24,9 +24,13 @@ RUN npm install -g "@zhafron/mcp-web-search@${MCP_WEB_SEARCH_VERSION}" \
|
||||
|
||||
ENV CHROME_PATH=/usr/bin/chromium \
|
||||
DEFAULT_SEARCH_PROVIDER=searxng \
|
||||
HOME=/tmp \
|
||||
HTTP_TIMEOUT=15000 \
|
||||
MAX_BYTES=${MCP_WEB_SEARCH_MAX_BYTES} \
|
||||
MAX_RESULTS=10 \
|
||||
SEARXNG_URL=http://searxng:8080
|
||||
SEARXNG_URL=http://searxng:8080 \
|
||||
XDG_CACHE_HOME=/tmp/.cache
|
||||
|
||||
USER node
|
||||
|
||||
ENTRYPOINT ["mcp-web-search"]
|
||||
|
||||
@@ -1,7 +1,14 @@
|
||||
# Assistant Setup
|
||||
|
||||
Context Kit supports any assistant that can run local stdio MCP servers. The
|
||||
included snippets cover Claude Code and OpenCode.
|
||||
Context Kit supports assistants that can run local stdio MCP servers, HTTP MCP
|
||||
servers, or both. The default transport split is simple:
|
||||
|
||||
- `context-web-search`: local stdio command.
|
||||
- `context-docs`: local HTTP MCP service.
|
||||
- `context-repomix`: local stdio command.
|
||||
|
||||
`bin/context-kit docs` is a stdio fallback for clients that cannot use HTTP MCP.
|
||||
The included snippets cover Claude Code and OpenCode.
|
||||
|
||||
## Claude Code
|
||||
|
||||
|
||||
@@ -7,7 +7,10 @@ Explicit environment variables win over `.env` values. The `.env` parser accepts
|
||||
simple `KEY=VALUE` lines for `CONTEXT_KIT_*` variables only; it does not execute
|
||||
shell code.
|
||||
|
||||
## Core Variables
|
||||
## User-Facing Variables
|
||||
|
||||
Only the variables below are part of the public configuration surface. Other
|
||||
`CONTEXT_KIT_*` variables used by scripts are release/test hooks and may change.
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
|---|---|---|
|
||||
@@ -68,14 +71,21 @@ Avoid `*`; the docs MCP is a local unauthenticated endpoint.
|
||||
|
||||
## Source Profiles
|
||||
|
||||
The docs MCP accepts one or more source files:
|
||||
The docs MCP accepts one or more source profile files:
|
||||
|
||||
```sh
|
||||
CONTEXT_KIT_DOCS_SOURCES="config/sources.default.txt config/sources.js.txt"
|
||||
```
|
||||
|
||||
Each source file is plain text. Blank lines and `#` comments are ignored.
|
||||
Entries may be absolute source-profile paths for private machine-local config.
|
||||
Source changes are loaded when the docs service starts. Run `bin/context-kit
|
||||
restart` after changing `CONTEXT_KIT_DOCS_SOURCES`; `bin/context-kit docs` only
|
||||
bridges stdio clients to the already-running service.
|
||||
|
||||
`CONTEXT_KIT_DOCS_SOURCES` may include absolute paths to private machine-local
|
||||
profile files. Each profile file is plain text; blank lines and `#` comments are
|
||||
ignored. Entries inside profile files must be URLs ending in `/llms.txt` or
|
||||
`/llms-full.txt`.
|
||||
|
||||
For local llms.txt files, place content under
|
||||
`CONTEXT_KIT_DOCS_LOCAL_SOURCES_DIR` and reference it as
|
||||
`http://127.0.0.1:8769/path/inside/local-sources/llms.txt` or another URL that
|
||||
|
||||
@@ -6,6 +6,7 @@ Context Kit is designed to be safe by default for local development.
|
||||
|
||||
- SearXNG is bound to `127.0.0.1` only.
|
||||
- No hosted API keys are required.
|
||||
- The web-search MCP image runs as the non-root `node` user.
|
||||
- Repomix mounts only the current project read-only.
|
||||
- Docs indexing stores data under `$HOME/.local/share/context-kit` unless you
|
||||
override it.
|
||||
|
||||
@@ -6,8 +6,21 @@
|
||||
bin/context-kit doctor
|
||||
```
|
||||
|
||||
This checks Docker, Compose, images, the Docker network, SearXNG health, and
|
||||
docs source configuration.
|
||||
This checks Docker, Compose, images, the Docker network, SearXNG health, docs
|
||||
HTTP readiness, and docs source configuration.
|
||||
|
||||
For release-grade MCP protocol checks, run:
|
||||
|
||||
```sh
|
||||
scripts/release-check
|
||||
```
|
||||
|
||||
Live provider checks are opt-in because search engines, remote docs, and model
|
||||
downloads can fail independently of this repo:
|
||||
|
||||
```sh
|
||||
CONTEXT_KIT_LIVE_CHECKS=1 scripts/release-check
|
||||
```
|
||||
|
||||
## SearXNG Is Not Responding
|
||||
|
||||
@@ -66,11 +79,13 @@ URL fetching uses the HTTP extractor path.
|
||||
|
||||
## Docs Indexing Is Slow
|
||||
|
||||
The first run downloads an embedding model and embeds every configured docs
|
||||
section. Keep default sources small, and add profiles only when you need them.
|
||||
The first `docs_query` or `docs_refresh` downloads an embedding model and
|
||||
embeds the requested docs sections lazily. Keep default sources small, and add
|
||||
profiles only when you need them.
|
||||
|
||||
Cloudflare and other large docs sets can take significantly longer than the
|
||||
default source profile.
|
||||
default source profile. Set `CONTEXT_KIT_DOCS_PREINDEX=1` only if you want
|
||||
startup to eagerly embed every configured source.
|
||||
|
||||
## Docs Tools Say Index Manager Not Initialized
|
||||
|
||||
@@ -79,7 +94,7 @@ If `docs_query` or `docs_refresh` returns `Index manager not initialized` while
|
||||
initialize its embedding model or Chroma database. Check the container logs:
|
||||
|
||||
```sh
|
||||
docker logs context-kit-docs-mcp
|
||||
docker compose -p "${CONTEXT_KIT_COMPOSE_PROJECT:-context-kit}" -f compose.yml logs docs-mcp
|
||||
```
|
||||
|
||||
A common cause is Docker creating the bind-mounted cache directories as `root`
|
||||
@@ -93,7 +108,7 @@ unable to open database file
|
||||
Fix ownership and restart:
|
||||
|
||||
```sh
|
||||
DATA_DIR="${CONTEXT_KIT_DATA_DIR:-$HOME/.local/share/context-kit}"
|
||||
DATA_DIR="${CONTEXT_KIT_DATA_DIR:-${HOME:?Set HOME or CONTEXT_KIT_DATA_DIR}/.local/share/context-kit}"
|
||||
sudo chown -R "$(id -u):$(id -g)" "$DATA_DIR/docs" "$DATA_DIR/models"
|
||||
bin/context-kit restart
|
||||
```
|
||||
|
||||
188
scripts/mcp-smoke-client.mjs
Normal file
188
scripts/mcp-smoke-client.mjs
Normal file
@@ -0,0 +1,188 @@
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
export function textFrom(result) {
|
||||
return (result.content || [])
|
||||
.filter(part => part.type === "text")
|
||||
.map(part => part.text)
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
export function requireToolSuccess(name, result) {
|
||||
if (result?.isError) {
|
||||
const payload = textFrom(result) || JSON.stringify(result);
|
||||
throw new Error(`${name} returned an error: ${payload.slice(0, 500)}`);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
export async function runSmoke({ usage, tmpPrefix, timeoutMs, clientInfo, scenario }) {
|
||||
const command = process.argv[2];
|
||||
const args = process.argv.slice(3);
|
||||
|
||||
if (!command) {
|
||||
throw new Error(usage);
|
||||
}
|
||||
|
||||
const client = new McpSmokeClient({ command, args, tmpPrefix });
|
||||
const timeout = setTimeout(async () => {
|
||||
await client.stop();
|
||||
console.error(`MCP smoke timed out. stderr: ${client.stderrTail(2000)}`);
|
||||
process.exit(1);
|
||||
}, timeoutMs);
|
||||
|
||||
try {
|
||||
await client.initialize(clientInfo);
|
||||
const result = await scenario(client);
|
||||
clearTimeout(timeout);
|
||||
await client.stop();
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
} catch (error) {
|
||||
clearTimeout(timeout);
|
||||
await client.stop();
|
||||
console.error(error.message);
|
||||
const stderr = client.stderrTail(4000);
|
||||
if (stderr) console.error(stderr);
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
class McpSmokeClient {
|
||||
constructor({ command, args, tmpPrefix }) {
|
||||
this.tmpDir = mkdtempSync(join(tmpdir(), tmpPrefix));
|
||||
this.cidFile = join(this.tmpDir, "container.cid");
|
||||
this.nextId = 1;
|
||||
this.pending = new Map();
|
||||
this.stdoutBuffer = "";
|
||||
this.stderrBuffer = "";
|
||||
this.exited = false;
|
||||
|
||||
this.child = spawn(command, args, {
|
||||
cwd: new URL("..", import.meta.url).pathname,
|
||||
env: { ...process.env, CONTEXT_KIT_DOCKER_CIDFILE: this.cidFile },
|
||||
stdio: ["pipe", "pipe", "pipe"]
|
||||
});
|
||||
|
||||
this.child.once("exit", (code, signal) => {
|
||||
this.exited = true;
|
||||
if (this.pending.size > 0) {
|
||||
const error = new Error(`MCP child exited before responding (code=${code}, signal=${signal}). stderr: ${this.stderrTail(2000)}`);
|
||||
for (const { reject } of this.pending.values()) reject(error);
|
||||
this.pending.clear();
|
||||
}
|
||||
});
|
||||
|
||||
this.child.stderr.on("data", chunk => {
|
||||
this.stderrBuffer += chunk.toString();
|
||||
});
|
||||
|
||||
this.child.stdout.on("data", chunk => this.handleStdout(chunk));
|
||||
}
|
||||
|
||||
stderrTail(length) {
|
||||
return this.stderrBuffer.slice(-length);
|
||||
}
|
||||
|
||||
handleStdout(chunk) {
|
||||
this.stdoutBuffer += chunk.toString();
|
||||
let newline;
|
||||
while ((newline = this.stdoutBuffer.indexOf("\n")) >= 0) {
|
||||
const line = this.stdoutBuffer.slice(0, newline).trim();
|
||||
this.stdoutBuffer = this.stdoutBuffer.slice(newline + 1);
|
||||
if (!line) continue;
|
||||
let message;
|
||||
try {
|
||||
message = JSON.parse(line);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (message.id && this.pending.has(message.id)) {
|
||||
const { resolve, reject } = this.pending.get(message.id);
|
||||
this.pending.delete(message.id);
|
||||
if (message.error) reject(new Error(JSON.stringify(message.error)));
|
||||
else resolve(message.result);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
request(method, params = {}) {
|
||||
if (this.exited) {
|
||||
return Promise.reject(new Error(`MCP child already exited. stderr: ${this.stderrTail(2000)}`));
|
||||
}
|
||||
const id = this.nextId++;
|
||||
this.child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", id, method, params })}\n`);
|
||||
return new Promise((resolve, reject) => this.pending.set(id, { resolve, reject }));
|
||||
}
|
||||
|
||||
notify(method, params = {}) {
|
||||
this.child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", method, params })}\n`);
|
||||
}
|
||||
|
||||
async initialize(clientInfo) {
|
||||
await this.request("initialize", {
|
||||
protocolVersion: "2024-11-05",
|
||||
capabilities: {},
|
||||
clientInfo
|
||||
});
|
||||
this.notify("notifications/initialized");
|
||||
}
|
||||
|
||||
async tools() {
|
||||
const listed = await this.request("tools/list");
|
||||
return new Set((listed.tools || []).map(tool => tool.name));
|
||||
}
|
||||
|
||||
async requireTools(names) {
|
||||
const toolNames = await this.tools();
|
||||
for (const name of names) {
|
||||
if (!toolNames.has(name)) throw new Error(`missing tool: ${name}`);
|
||||
}
|
||||
return toolNames;
|
||||
}
|
||||
|
||||
callTool(name, args = {}) {
|
||||
return this.request("tools/call", { name, arguments: args });
|
||||
}
|
||||
|
||||
stopContainer() {
|
||||
if (!existsSync(this.cidFile)) return;
|
||||
const containerId = readFileSync(this.cidFile, "utf8").trim();
|
||||
if (!containerId) return;
|
||||
spawnSync("docker", ["stop", containerId], { stdio: "ignore" });
|
||||
}
|
||||
|
||||
stop() {
|
||||
return new Promise(resolve => {
|
||||
if (this.exited) {
|
||||
this.stopContainer();
|
||||
rmSync(this.tmpDir, { recursive: true, force: true });
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
|
||||
let stopTimer;
|
||||
let termTimer;
|
||||
let killTimer;
|
||||
this.child.once("exit", () => {
|
||||
this.stopContainer();
|
||||
clearTimeout(stopTimer);
|
||||
clearTimeout(termTimer);
|
||||
clearTimeout(killTimer);
|
||||
rmSync(this.tmpDir, { recursive: true, force: true });
|
||||
resolve();
|
||||
});
|
||||
|
||||
stopTimer = setTimeout(() => this.stopContainer(), 1000);
|
||||
termTimer = setTimeout(() => {
|
||||
if (!this.exited) this.child.kill("SIGTERM");
|
||||
}, 3000);
|
||||
killTimer = setTimeout(() => {
|
||||
if (!this.exited) this.child.kill("SIGKILL");
|
||||
}, 6000);
|
||||
|
||||
this.child.stdin.end();
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -5,7 +5,29 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "${ROOT}"
|
||||
|
||||
tmp_dir="$(mktemp -d)"
|
||||
pick_port() {
|
||||
python - <<'PY'
|
||||
import socket
|
||||
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||
sock.bind(("127.0.0.1", 0))
|
||||
print(sock.getsockname()[1])
|
||||
PY
|
||||
}
|
||||
|
||||
release_id="release-$$"
|
||||
export CONTEXT_KIT_COMPOSE_PROJECT="context-kit-${release_id}"
|
||||
export CONTEXT_KIT_DATA_DIR="${tmp_dir}/data"
|
||||
export CONTEXT_KIT_SEARXNG_PORT="$(pick_port)"
|
||||
export CONTEXT_KIT_DOCS_PORT="$(pick_port)"
|
||||
export CONTEXT_KIT_DOCS_SOURCES="config/sources.default.txt"
|
||||
export CONTEXT_KIT_DOCS_LOCAL_SOURCES_DIR="${tmp_dir}/local-sources"
|
||||
export CONTEXT_KIT_WEB_SEARCH_IMAGE="context-kit/web-search-mcp:${release_id}"
|
||||
export CONTEXT_KIT_DOCS_IMAGE="context-kit/docs-mcp:${release_id}"
|
||||
|
||||
cleanup() {
|
||||
docker compose -p "${CONTEXT_KIT_COMPOSE_PROJECT}" -f compose.yml down -v --remove-orphans >/dev/null 2>&1 || true
|
||||
docker image rm "${CONTEXT_KIT_WEB_SEARCH_IMAGE}" "${CONTEXT_KIT_DOCS_IMAGE}" >/dev/null 2>&1 || true
|
||||
rm -rf "${tmp_dir}"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
@@ -32,19 +54,47 @@ assert_redaction_check_does_not_disclose_matches() {
|
||||
fi
|
||||
}
|
||||
|
||||
assert_web_search_image() {
|
||||
docker run --rm --entrypoint node \
|
||||
-e EXPECTED_MAX_BYTES="${CONTEXT_KIT_WEB_SEARCH_MAX_BYTES:-52428800}" \
|
||||
"${CONTEXT_KIT_WEB_SEARCH_IMAGE}" \
|
||||
-e '
|
||||
const fs = require("node:fs");
|
||||
const expected = Number(process.env.EXPECTED_MAX_BYTES) || 0;
|
||||
const actual = Number(process.env.MAX_BYTES) || 0;
|
||||
if (process.getuid && process.getuid() === 0) process.exit(1);
|
||||
if (actual !== expected) process.exit(1);
|
||||
|
||||
const serverPath = "/usr/local/lib/node_modules/@zhafron/mcp-web-search/dist/src/server.js";
|
||||
const server = fs.readFileSync(serverPath, "utf8");
|
||||
if (!server.includes("max_download_bytes: z.number().int().min(1).max(MAX_BYTES).optional()")) process.exit(1);
|
||||
|
||||
const bingPath = "/usr/local/lib/node_modules/@zhafron/mcp-web-search/dist/src/providers/bing.js";
|
||||
const bing = fs.readFileSync(bingPath, "utf8");
|
||||
if (!bing.includes("Context Kit override for @zhafron/mcp-web-search 1.3.0")) process.exit(1);
|
||||
if (!bing.includes("waitForSelector")) process.exit(1);
|
||||
if (!bing.includes("decodeBingRedirect")) process.exit(1);
|
||||
' >/dev/null
|
||||
|
||||
docker run --rm --entrypoint /usr/bin/test \
|
||||
"${CONTEXT_KIT_WEB_SEARCH_IMAGE}" \
|
||||
-x "${CONTEXT_KIT_WEB_SEARCH_CHROME_PATH:-/usr/bin/chromium}"
|
||||
}
|
||||
|
||||
git diff --check HEAD
|
||||
git show --check --format= HEAD >/dev/null
|
||||
git ls-files --cached --error-unmatch \
|
||||
docker/web-search/patch-mcp-web-search.mjs \
|
||||
docker/web-search/overrides/bing.js \
|
||||
docker/docs/constraints.txt \
|
||||
scripts/mcp-smoke-client.mjs \
|
||||
scripts/smoke-web-search.mjs \
|
||||
scripts/smoke-docs.mjs \
|
||||
scripts/release-check >/dev/null
|
||||
bash -n bin/context-kit
|
||||
bash -n scripts/release-check
|
||||
sh -n docker/docs/entrypoint.sh
|
||||
check_node docker/web-search/patch-mcp-web-search.mjs docker/web-search/overrides/bing.js scripts/smoke-web-search.mjs scripts/smoke-docs.mjs
|
||||
check_node docker/web-search/patch-mcp-web-search.mjs docker/web-search/overrides/bing.js scripts/mcp-smoke-client.mjs scripts/smoke-web-search.mjs scripts/smoke-docs.mjs
|
||||
|
||||
node -e 'const fs=require("node:fs"); JSON.parse(fs.readFileSync("snippets/opencode.json", "utf8")); JSON.parse(fs.readFileSync("snippets/claude.mcp.json", "utf8"));'
|
||||
bin/context-kit install opencode > "${tmp_dir}/opencode.json"
|
||||
@@ -60,13 +110,14 @@ bin/context-kit redaction-check "${tmp_dir}/opencode.json" "${tmp_dir}/claude.js
|
||||
assert_redaction_check_does_not_disclose_matches
|
||||
|
||||
bin/context-kit redaction-check
|
||||
docker compose -p context-kit -f compose.yml config >/dev/null
|
||||
if env -u HOME docker compose --env-file /dev/null -p context-kit-release-home-check -f compose.yml config >"${tmp_dir}/compose-no-home.out" 2>"${tmp_dir}/compose-no-home.err"; then
|
||||
docker compose -p "${CONTEXT_KIT_COMPOSE_PROJECT}" -f compose.yml config >/dev/null
|
||||
if env -u HOME -u CONTEXT_KIT_DATA_DIR -u CONTEXT_KIT_DOCS_LOCAL_SOURCES_DIR docker compose --env-file /dev/null -p context-kit-release-home-check -f compose.yml config >"${tmp_dir}/compose-no-home.out" 2>"${tmp_dir}/compose-no-home.err"; then
|
||||
printf 'compose config unexpectedly succeeded without HOME or CONTEXT_KIT_DATA_DIR\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
CONTEXT_KIT_DATA_DIR="${tmp_dir}/compose-data" env -u HOME docker compose --env-file /dev/null -p context-kit-release-home-check -f compose.yml config >/dev/null
|
||||
bin/context-kit build
|
||||
assert_web_search_image
|
||||
bin/context-kit restart
|
||||
bin/context-kit doctor
|
||||
node scripts/smoke-web-search.mjs bin/context-kit web-search
|
||||
|
||||
@@ -1,170 +1,42 @@
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { requireToolSuccess, runSmoke } from "./mcp-smoke-client.mjs";
|
||||
|
||||
const command = process.argv[2];
|
||||
const args = process.argv.slice(3);
|
||||
const live = process.env.CONTEXT_KIT_LIVE_CHECKS === "1";
|
||||
|
||||
if (!command) {
|
||||
throw new Error("usage: node scripts/smoke-docs.mjs <command> [args...]");
|
||||
}
|
||||
runSmoke({
|
||||
usage: "usage: node scripts/smoke-docs.mjs <command> [args...]",
|
||||
tmpPrefix: "context-kit-docs-smoke-",
|
||||
timeoutMs: 300000,
|
||||
clientInfo: { name: "context-kit-docs-smoke", version: "0.0.0" },
|
||||
scenario: async client => {
|
||||
const toolNames = await client.requireTools(["docs_query", "docs_sources"]);
|
||||
|
||||
const tmpDir = mkdtempSync(join(tmpdir(), "context-kit-docs-smoke-"));
|
||||
const cidFile = join(tmpDir, "container.cid");
|
||||
|
||||
const child = spawn(command, args, {
|
||||
cwd: new URL("..", import.meta.url).pathname,
|
||||
env: { ...process.env, CONTEXT_KIT_DOCKER_CIDFILE: cidFile },
|
||||
stdio: ["pipe", "pipe", "pipe"]
|
||||
});
|
||||
|
||||
let nextId = 1;
|
||||
const pending = new Map();
|
||||
let stdoutBuffer = "";
|
||||
let stderrBuffer = "";
|
||||
let childExited = false;
|
||||
|
||||
child.once("exit", (code, signal) => {
|
||||
childExited = true;
|
||||
if (pending.size > 0) {
|
||||
const error = new Error(`MCP child exited before responding (code=${code}, signal=${signal}). stderr: ${stderrBuffer.slice(-2000)}`);
|
||||
for (const { reject } of pending.values()) reject(error);
|
||||
pending.clear();
|
||||
}
|
||||
});
|
||||
|
||||
function stopChild() {
|
||||
return new Promise(resolve => {
|
||||
if (childExited) {
|
||||
stopContainer();
|
||||
rmSync(tmpDir, { recursive: true, force: true });
|
||||
resolve();
|
||||
return;
|
||||
const sources = requireToolSuccess("docs_sources", await client.callTool("docs_sources"));
|
||||
if (!Array.isArray(sources?.structuredContent?.result)) {
|
||||
const sourcesText = JSON.stringify(sources);
|
||||
throw new Error(`docs_sources returned unexpected payload: ${sourcesText.slice(0, 500)}`);
|
||||
}
|
||||
|
||||
child.stdin.end();
|
||||
const stopTimer = setTimeout(() => {
|
||||
stopContainer();
|
||||
}, 1000);
|
||||
const termTimer = setTimeout(() => {
|
||||
if (!childExited) child.kill("SIGTERM");
|
||||
}, 3000);
|
||||
const killTimer = setTimeout(() => {
|
||||
if (!childExited) child.kill("SIGKILL");
|
||||
}, 6000);
|
||||
const result = {
|
||||
tools: Array.from(toolNames).sort(),
|
||||
docs_sources: "pass"
|
||||
};
|
||||
|
||||
child.once("exit", () => {
|
||||
stopContainer();
|
||||
clearTimeout(stopTimer);
|
||||
clearTimeout(termTimer);
|
||||
clearTimeout(killTimer);
|
||||
rmSync(tmpDir, { recursive: true, force: true });
|
||||
resolve();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function stopContainer() {
|
||||
if (!existsSync(cidFile)) return;
|
||||
const containerId = readFileSync(cidFile, "utf8").trim();
|
||||
if (!containerId) return;
|
||||
spawnSync("docker", ["stop", containerId], { stdio: "ignore" });
|
||||
}
|
||||
|
||||
const timeout = setTimeout(async () => {
|
||||
await stopChild();
|
||||
console.error(`Docs MCP smoke timed out. stderr: ${stderrBuffer.slice(-2000)}`);
|
||||
process.exit(1);
|
||||
}, 300000);
|
||||
|
||||
child.stderr.on("data", chunk => {
|
||||
stderrBuffer += chunk.toString();
|
||||
});
|
||||
|
||||
child.stdout.on("data", chunk => {
|
||||
stdoutBuffer += chunk.toString();
|
||||
let newline;
|
||||
while ((newline = stdoutBuffer.indexOf("\n")) >= 0) {
|
||||
const line = stdoutBuffer.slice(0, newline).trim();
|
||||
stdoutBuffer = stdoutBuffer.slice(newline + 1);
|
||||
if (!line) continue;
|
||||
let message;
|
||||
try {
|
||||
message = JSON.parse(line);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (message.id && pending.has(message.id)) {
|
||||
const { resolve, reject } = pending.get(message.id);
|
||||
pending.delete(message.id);
|
||||
if (message.error) reject(new Error(JSON.stringify(message.error)));
|
||||
else resolve(message.result);
|
||||
if (live) {
|
||||
const query = requireToolSuccess("docs_query", await client.callTool("docs_query", {
|
||||
query: "Model Context Protocol documentation",
|
||||
limit: 3,
|
||||
auto_retrieve: true,
|
||||
auto_retrieve_threshold: 0.1,
|
||||
auto_retrieve_limit: 1,
|
||||
max_bytes: 12000
|
||||
}));
|
||||
const queryText = JSON.stringify(query);
|
||||
if (!queryText.includes("search_results") && !queryText.includes("Model Context Protocol")) {
|
||||
throw new Error(`docs_query returned unexpected payload: ${queryText.slice(0, 500)}`);
|
||||
}
|
||||
result.docs_query = "pass";
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
});
|
||||
|
||||
function request(method, params = {}) {
|
||||
if (childExited) {
|
||||
return Promise.reject(new Error(`MCP child already exited. stderr: ${stderrBuffer.slice(-2000)}`));
|
||||
}
|
||||
const id = nextId++;
|
||||
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", id, method, params })}\n`);
|
||||
return new Promise((resolve, reject) => pending.set(id, { resolve, reject }));
|
||||
}
|
||||
|
||||
function notify(method, params = {}) {
|
||||
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", method, params })}\n`);
|
||||
}
|
||||
|
||||
async function callTool(name, args = {}) {
|
||||
return request("tools/call", { name, arguments: args });
|
||||
}
|
||||
|
||||
try {
|
||||
await request("initialize", {
|
||||
protocolVersion: "2024-11-05",
|
||||
capabilities: {},
|
||||
clientInfo: { name: "context-kit-docs-smoke", version: "0.0.0" }
|
||||
});
|
||||
notify("notifications/initialized");
|
||||
|
||||
const listed = await request("tools/list");
|
||||
const toolNames = new Set((listed.tools || []).map(tool => tool.name));
|
||||
for (const name of ["docs_query", "docs_sources"]) {
|
||||
if (!toolNames.has(name)) throw new Error(`missing tool: ${name}`);
|
||||
}
|
||||
|
||||
const sources = await callTool("docs_sources");
|
||||
const sourcesText = JSON.stringify(sources);
|
||||
if (sources.isError) {
|
||||
throw new Error(`docs_sources returned an error: ${sourcesText.slice(0, 500)}`);
|
||||
}
|
||||
|
||||
const query = await callTool("docs_query", {
|
||||
query: "Model Context Protocol documentation",
|
||||
limit: 3,
|
||||
auto_retrieve: true,
|
||||
auto_retrieve_threshold: 0.1,
|
||||
auto_retrieve_limit: 1,
|
||||
max_bytes: 12000
|
||||
});
|
||||
const queryText = JSON.stringify(query);
|
||||
if (!queryText.includes("search_results") && !queryText.includes("Model Context Protocol")) {
|
||||
throw new Error(`docs_query returned unexpected payload: ${queryText.slice(0, 500)}`);
|
||||
}
|
||||
|
||||
clearTimeout(timeout);
|
||||
await stopChild();
|
||||
console.log(JSON.stringify({
|
||||
tools: Array.from(toolNames).sort(),
|
||||
docs_sources: "pass",
|
||||
docs_query: "pass"
|
||||
}, null, 2));
|
||||
} catch (error) {
|
||||
clearTimeout(timeout);
|
||||
await stopChild();
|
||||
console.error(error.message);
|
||||
if (stderrBuffer) console.error(stderrBuffer.slice(-4000));
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
@@ -1,197 +1,63 @@
|
||||
import { spawn, spawnSync } from "node:child_process";
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { requireToolSuccess, runSmoke, textFrom } from "./mcp-smoke-client.mjs";
|
||||
|
||||
const command = process.argv[2];
|
||||
const args = process.argv.slice(3);
|
||||
const live = process.env.CONTEXT_KIT_LIVE_CHECKS === "1";
|
||||
|
||||
if (!command) {
|
||||
throw new Error("usage: node scripts/smoke-web-search.mjs <command> [args...]");
|
||||
}
|
||||
runSmoke({
|
||||
usage: "usage: node scripts/smoke-web-search.mjs <command> [args...]",
|
||||
tmpPrefix: "context-kit-web-search-smoke-",
|
||||
timeoutMs: 120000,
|
||||
clientInfo: { name: "context-kit-web-search-smoke", version: "0.0.0" },
|
||||
scenario: async client => {
|
||||
const toolNames = await client.requireTools(["search_web", "fetch_url"]);
|
||||
|
||||
const tmpDir = mkdtempSync(join(tmpdir(), "context-kit-web-search-smoke-"));
|
||||
const cidFile = join(tmpDir, "container.cid");
|
||||
|
||||
const child = spawn(command, args, {
|
||||
cwd: new URL("..", import.meta.url).pathname,
|
||||
env: { ...process.env, CONTEXT_KIT_DOCKER_CIDFILE: cidFile },
|
||||
stdio: ["pipe", "pipe", "pipe"]
|
||||
});
|
||||
|
||||
let nextId = 1;
|
||||
const pending = new Map();
|
||||
let stdoutBuffer = "";
|
||||
let stderrBuffer = "";
|
||||
let childExited = false;
|
||||
|
||||
child.once("exit", (code, signal) => {
|
||||
childExited = true;
|
||||
if (pending.size > 0) {
|
||||
const error = new Error(`MCP child exited before responding (code=${code}, signal=${signal}). stderr: ${stderrBuffer.slice(-2000)}`);
|
||||
for (const { reject } of pending.values()) reject(error);
|
||||
pending.clear();
|
||||
}
|
||||
});
|
||||
|
||||
function stopChild() {
|
||||
return new Promise(resolve => {
|
||||
if (childExited) {
|
||||
stopContainer();
|
||||
rmSync(tmpDir, { recursive: true, force: true });
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
|
||||
child.stdin.end();
|
||||
const stopTimer = setTimeout(() => {
|
||||
stopContainer();
|
||||
}, 1000);
|
||||
const termTimer = setTimeout(() => {
|
||||
if (!childExited) child.kill("SIGTERM");
|
||||
}, 3000);
|
||||
const killTimer = setTimeout(() => {
|
||||
if (!childExited) child.kill("SIGKILL");
|
||||
}, 6000);
|
||||
|
||||
child.once("exit", () => {
|
||||
stopContainer();
|
||||
clearTimeout(stopTimer);
|
||||
clearTimeout(termTimer);
|
||||
clearTimeout(killTimer);
|
||||
rmSync(tmpDir, { recursive: true, force: true });
|
||||
resolve();
|
||||
const localResult = await client.callTool("fetch_url", {
|
||||
url: "http://127.0.0.1:1/",
|
||||
max_download_bytes: 52428800
|
||||
});
|
||||
});
|
||||
}
|
||||
const localBlocked = Boolean(localResult.isError) && textFrom(localResult).includes("Blocked localhost/private URL");
|
||||
if (!localBlocked) throw new Error("localhost/private URL was not blocked as expected");
|
||||
|
||||
function stopContainer() {
|
||||
if (!existsSync(cidFile)) return;
|
||||
const containerId = readFileSync(cidFile, "utf8").trim();
|
||||
if (!containerId) return;
|
||||
spawnSync("docker", ["stop", containerId], { stdio: "ignore" });
|
||||
}
|
||||
const result = {
|
||||
tools: Array.from(toolNames).sort(),
|
||||
localhost_guard: "pass"
|
||||
};
|
||||
|
||||
const timeout = setTimeout(async () => {
|
||||
await stopChild();
|
||||
console.error(`MCP smoke timed out. stderr: ${stderrBuffer.slice(-2000)}`);
|
||||
process.exit(1);
|
||||
}, 120000);
|
||||
if (live) {
|
||||
const searxng = textFrom(requireToolSuccess("search_web/searxng", await client.callTool("search_web", {
|
||||
q: "Model Context Protocol",
|
||||
limit: 2,
|
||||
provider: "searxng"
|
||||
})));
|
||||
if (!searxng.includes("Model")) throw new Error(`SearXNG smoke returned unexpected text: ${searxng.slice(0, 500)}`);
|
||||
|
||||
child.stderr.on("data", chunk => {
|
||||
stderrBuffer += chunk.toString();
|
||||
});
|
||||
const bing = textFrom(requireToolSuccess("search_web/bing", await client.callTool("search_web", {
|
||||
q: "Model Context Protocol",
|
||||
limit: 2,
|
||||
provider: "bing"
|
||||
})));
|
||||
if (!bing.includes("Model")) throw new Error(`Bing smoke returned unexpected text: ${bing.slice(0, 500)}`);
|
||||
|
||||
child.stdout.on("data", chunk => {
|
||||
stdoutBuffer += chunk.toString();
|
||||
let newline;
|
||||
while ((newline = stdoutBuffer.indexOf("\n")) >= 0) {
|
||||
const line = stdoutBuffer.slice(0, newline).trim();
|
||||
stdoutBuffer = stdoutBuffer.slice(newline + 1);
|
||||
if (!line) continue;
|
||||
let message;
|
||||
try {
|
||||
message = JSON.parse(line);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (message.id && pending.has(message.id)) {
|
||||
const { resolve, reject } = pending.get(message.id);
|
||||
pending.delete(message.id);
|
||||
if (message.error) reject(new Error(JSON.stringify(message.error)));
|
||||
else resolve(message.result);
|
||||
const fetch = textFrom(requireToolSuccess("fetch_url/http", await client.callTool("fetch_url", {
|
||||
url: "https://example.com/",
|
||||
format: "markdown",
|
||||
max_download_bytes: 52428800
|
||||
})));
|
||||
if (!fetch.includes("Example Domain")) throw new Error(`fetch smoke returned unexpected text: ${fetch.slice(0, 500)}`);
|
||||
|
||||
const browserFetch = textFrom(requireToolSuccess("fetch_url/browser", await client.callTool("fetch_url", {
|
||||
url: "https://example.com/",
|
||||
format: "markdown",
|
||||
engine: "browser",
|
||||
max_download_bytes: 52428800
|
||||
})));
|
||||
if (!browserFetch.includes("Example Domain")) throw new Error(`browser fetch smoke returned unexpected text: ${browserFetch.slice(0, 500)}`);
|
||||
|
||||
result.searxng = "pass";
|
||||
result.bing = "pass";
|
||||
result.fetch_url = "pass";
|
||||
result.fetch_url_browser_engine_currently_http = "pass";
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
});
|
||||
|
||||
function request(method, params = {}) {
|
||||
if (childExited) {
|
||||
return Promise.reject(new Error(`MCP child already exited. stderr: ${stderrBuffer.slice(-2000)}`));
|
||||
}
|
||||
const id = nextId++;
|
||||
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", id, method, params })}\n`);
|
||||
return new Promise((resolve, reject) => pending.set(id, { resolve, reject }));
|
||||
}
|
||||
|
||||
function notify(method, params = {}) {
|
||||
child.stdin.write(`${JSON.stringify({ jsonrpc: "2.0", method, params })}\n`);
|
||||
}
|
||||
|
||||
function textFrom(result) {
|
||||
return (result.content || [])
|
||||
.filter(part => part.type === "text")
|
||||
.map(part => part.text)
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
async function callTool(name, args = {}) {
|
||||
return request("tools/call", { name, arguments: args });
|
||||
}
|
||||
|
||||
try {
|
||||
await request("initialize", {
|
||||
protocolVersion: "2024-11-05",
|
||||
capabilities: {},
|
||||
clientInfo: { name: "context-kit-smoke", version: "0.0.0" }
|
||||
});
|
||||
notify("notifications/initialized");
|
||||
|
||||
const listed = await request("tools/list");
|
||||
const toolNames = new Set((listed.tools || []).map(tool => tool.name));
|
||||
for (const name of ["search_web", "fetch_url"]) {
|
||||
if (!toolNames.has(name)) throw new Error(`missing tool: ${name}`);
|
||||
}
|
||||
|
||||
const searxng = textFrom(await callTool("search_web", {
|
||||
q: "Model Context Protocol",
|
||||
limit: 2,
|
||||
provider: "searxng"
|
||||
}));
|
||||
if (!searxng.includes("Model")) throw new Error(`SearXNG smoke returned unexpected text: ${searxng.slice(0, 500)}`);
|
||||
|
||||
const bing = textFrom(await callTool("search_web", {
|
||||
q: "Model Context Protocol",
|
||||
limit: 2,
|
||||
provider: "bing"
|
||||
}));
|
||||
if (!bing.includes("Model")) throw new Error(`Bing smoke returned unexpected text: ${bing.slice(0, 500)}`);
|
||||
|
||||
const fetch = textFrom(await callTool("fetch_url", {
|
||||
url: "https://example.com/",
|
||||
format: "markdown",
|
||||
max_download_bytes: 52428800
|
||||
}));
|
||||
if (!fetch.includes("Example Domain")) throw new Error(`fetch smoke returned unexpected text: ${fetch.slice(0, 500)}`);
|
||||
|
||||
const browserFetch = textFrom(await callTool("fetch_url", {
|
||||
url: "https://example.com/",
|
||||
format: "markdown",
|
||||
engine: "browser",
|
||||
max_download_bytes: 52428800
|
||||
}));
|
||||
if (!browserFetch.includes("Example Domain")) throw new Error(`browser fetch smoke returned unexpected text: ${browserFetch.slice(0, 500)}`);
|
||||
|
||||
const localResult = await callTool("fetch_url", {
|
||||
url: "http://127.0.0.1:1/",
|
||||
max_download_bytes: 52428800
|
||||
});
|
||||
const localBlocked = Boolean(localResult.isError) && textFrom(localResult).includes("Blocked localhost/private URL");
|
||||
if (!localBlocked) throw new Error("localhost/private URL was not blocked as expected");
|
||||
|
||||
clearTimeout(timeout);
|
||||
await stopChild();
|
||||
console.log(JSON.stringify({
|
||||
tools: Array.from(toolNames).sort(),
|
||||
searxng: "pass",
|
||||
bing: "pass",
|
||||
fetch_url: "pass",
|
||||
fetch_url_browser_engine_currently_http: "pass",
|
||||
localhost_guard: "pass"
|
||||
}, null, 2));
|
||||
} catch (error) {
|
||||
clearTimeout(timeout);
|
||||
await stopChild();
|
||||
console.error(error.message);
|
||||
if (stderrBuffer) console.error(stderrBuffer.slice(-4000));
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user