feat: run docs-mcp as a long-lived shared HTTP service
context-docs was previously spawned per call as a fresh stdio container,
which meant every MCP request paid full cold-start cost (embedding model
load + Chroma open) and concurrent clients raced for the same Chroma
writer. The 50+ orphan container build-up I saw during the publish audit
was the visible symptom.
This refactor runs docs-mcp as one long-lived service:
- compose: docs-mcp leaves the 'mcp' profile, gets container_name,
restart: unless-stopped, healthcheck, and a host port (127.0.0.1:8776
by default). Runs as the host UID/GID so bind mounts don't end up
root-owned.
- docker image: adds mcp-proxy (0.12.0) and an entrypoint that fronts
llms-txt-mcp's stdio as Streamable HTTP. Reads sources from a flat
file mounted at /etc/context-kit/docs-sources.txt. Disables eager
preindex by default; callers refresh on demand via the docs_refresh
tool. Set CONTEXT_KIT_DOCS_PREINDEX=1 to restore eager behavior.
- bin/context-kit: 'start' brings up the docs service alongside SearXNG,
generates the sources file from CONTEXT_KIT_DOCS_SOURCES, and waits
for the HTTP endpoint to become ready (up to 180s for first-run model
download). 'docs' still works for stdio-only clients but is now a
thin mcp-proxy bridge onto the shared HTTP service. 'doctor' and
'status' both surface the new endpoint.
- install snippets: context-docs is now 'type: remote'/'type: http'
pointing at ${CONTEXT_KIT_DOCS_HTTP_URL}. HTTP-capable MCP clients
bypass the bridge entirely. snippets/*.json and the install command
output stay byte-identical.
- docs and .env.example updated for new vars (CONTEXT_KIT_DOCS_PORT,
CONTEXT_KIT_DOCS_HTTP_URL, CONTEXT_KIT_DOCS_PREINDEX) and the new
24h TTL default (down from 7d; the long-lived service makes shorter
defaults cheap).
Verified end-to-end:
- compose config -q, bash -n, sh -n all clean
- HTTP /status returns 200
- stdio bridge returns initialize + tools/list with the same 3 tools
(docs_sources, docs_refresh, docs_query)
- doctor passes all 10 checks including the new HTTP probe
- web-search and repomix MCP handshakes still work
- redaction-check clean
- install JSON valid for both targets + --absolute
This commit is contained in:
53
docker/docs/entrypoint.sh
Normal file
53
docker/docs/entrypoint.sh
Normal file
@@ -0,0 +1,53 @@
|
||||
#!/bin/sh
|
||||
# context-kit docs-mcp entrypoint.
|
||||
#
|
||||
# Bridges llms-txt-mcp (stdio-only) to Streamable HTTP via mcp-proxy so that
|
||||
# multiple clients share a single long-lived indexer instead of each spawning
|
||||
# their own container (and racing on the same Chroma store).
|
||||
#
|
||||
# Sources are read from $DOCS_MCP_SOURCES_FILE (one URL per line; `#` comments
|
||||
# and blank lines are allowed). Everything else is configured via env vars
|
||||
# with sensible defaults so this image works standalone too.
|
||||
|
||||
set -eu
|
||||
|
||||
sources_file="${DOCS_MCP_SOURCES_FILE:-/etc/context-kit/docs-sources.txt}"
|
||||
|
||||
if [ ! -r "$sources_file" ]; then
|
||||
echo "docs-mcp: sources file not readable: $sources_file" >&2
|
||||
echo "docs-mcp: set DOCS_MCP_SOURCES_FILE or mount one at that path." >&2
|
||||
exit 64
|
||||
fi
|
||||
|
||||
# Strip comments and blank lines, then collapse whitespace into a flat list.
|
||||
sources=$(grep -vE '^[[:space:]]*(#|$)' "$sources_file" | tr -s '[:space:]' '\n' | grep -v '^$' || true)
|
||||
|
||||
if [ -z "$sources" ]; then
|
||||
echo "docs-mcp: no sources found in $sources_file after stripping comments/blanks" >&2
|
||||
exit 64
|
||||
fi
|
||||
|
||||
# By default llms-txt-mcp 0.2.0 re-embeds every source on launch (the actual
|
||||
# default is a background preindex, --no-preindex only disables the foreground
|
||||
# variant). On a long-lived container that just wastes ~5 min of CPU per
|
||||
# restart, so we disable BOTH and let the caller use `docs_refresh` on demand.
|
||||
# Set DOCS_MCP_PREINDEX=1 to restore the eager behavior.
|
||||
preindex_flag="--no-preindex --no-background-preindex"
|
||||
if [ "${DOCS_MCP_PREINDEX:-0}" = "1" ]; then
|
||||
preindex_flag=""
|
||||
fi
|
||||
|
||||
# shellcheck disable=SC2086 # intentional word-splitting on $sources / $preindex_flag
|
||||
exec mcp-proxy \
|
||||
--host "${DOCS_MCP_HTTP_HOST:-0.0.0.0}" \
|
||||
--port "${DOCS_MCP_HTTP_PORT:-8000}" \
|
||||
--pass-environment \
|
||||
--allow-origin "${DOCS_MCP_ALLOW_ORIGIN:-*}" \
|
||||
-- \
|
||||
llms-txt-mcp \
|
||||
--store-path /data \
|
||||
--ttl "${DOCS_MCP_TTL:-24h}" \
|
||||
--max-get-bytes "${DOCS_MCP_MAX_GET_BYTES:-75000}" \
|
||||
--embed-model "${DOCS_MCP_EMBED_MODEL:-BAAI/bge-small-en-v1.5}" \
|
||||
$preindex_flag \
|
||||
$sources
|
||||
Reference in New Issue
Block a user