ajaynomics/opencode-compat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Certify alpha7 compatibility candidates

Browse Source
This commit is contained in:
Ajay Krishnan
2026-07-18 15:10:59 -07:00
parent 49604f18bd
commit 4bd552f389
20 changed files with 599 additions and 121 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/candidate.yml vendored
View File

@@ -20,7 +20,7 @@ jobs:
- uses: ruby/setup-ruby@v1 - uses: ruby/setup-ruby@v1
with: with:
ruby-version: "3.4" ruby-version: "3.4"
- run: ruby test/repository_test.rb && ruby test/runtime_tuple_promoter_test.rb - run: ruby test/repository_test.rb && ruby test/runtime_tuple_promoter_test.rb && ruby test/exact_live_contract_test.rb
prepare: prepare:
runs-on: ubuntu-latest runs-on: ubuntu-latest

12
README.md
View File

@@ -18,6 +18,9 @@ prompts, credentials, sessions, or user data belong here.
neither workflow deploys or changes a consumer. neither workflow deploys or changes a consumer.
- Each consumer retains both `current` and `previous` certified tuples so a - Each consumer retains both `current` and `previous` certified tuples so a
rollback restores the gem and server together. rollback restores the gem and server together.
- A failing bootstrap baseline is never relabeled as a certified `previous`
tuple. Promotion stays blocked until a distinct rollback consumer commit is
pinned to a passing client/runtime tuple and canaried.
## What is covered ## What is covered
@@ -42,9 +45,10 @@ Prerequisites are Ruby 3.2+, Python 3, `jq`, and Docker.
```sh ```sh
ruby test/repository_test.rb ruby test/repository_test.rb
ruby test/runtime_tuple_promoter_test.rb ruby test/runtime_tuple_promoter_test.rb
OPENCODE_RUBY_PATH=/data/projects/opencode-ruby-alpha6 \ ruby test/exact_live_contract_test.rb
OPENCODE_RUBY_PATH=/path/to/opencode-ruby-at-78b6f9c9e9c7d58b699af1c3c17764acd33de798 \
ruby ruby/opencode_ruby_fixture_contract.rb ruby ruby/opencode_ruby_fixture_contract.rb
OPENCODE_RUBY_PATH=/data/projects/opencode-ruby-alpha6 \ OPENCODE_RUBY_PATH=/path/to/opencode-ruby-at-78b6f9c9e9c7d58b699af1c3c17764acd33de798 \
OPENCODE_IMAGE='ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a' \ OPENCODE_IMAGE='ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a' \
scripts/run_image_contract.sh scripts/run_image_contract.sh
``` ```
@@ -52,7 +56,9 @@ OPENCODE_IMAGE='ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca
The live contract starts a deterministic local OpenAI-compatible model stub and The live contract starts a deterministic local OpenAI-compatible model stub and
an isolated OpenCode container. It creates a session, subscribes, submits an an isolated OpenCode container. It creates a session, subscribes, submits an
async prompt, observes terminal SSE, fetches the authoritative exchange, and async prompt, observes terminal SSE, fetches the authoritative exchange, and
deletes the session. It never calls an external model provider. deletes the session. Passing requires the authoritative final text to equal the
expected text byte-for-byte and the deterministic model stub to observe exactly
one request. It never calls an external model provider.
## Adding an upstream release ## Adding an upstream release

30
docs/certification.md
View File

@@ -18,6 +18,11 @@ The gate requires:
Health-only probes do not certify a tuple. Health-only probes do not certify a tuple.
The shared live probe is intentionally strict: `full_text` must equal the
expected text byte-for-byte and the deterministic model must receive exactly
one request. A response that merely contains the expected text, including a
duplicated `compat-ok\n\ncompat-ok`, fails.
## Promotion ## Promotion
Promotion is a reviewed manifest change that moves the old `current` tuple to Promotion is a reviewed manifest change that moves the old `current` tuple to
@@ -78,10 +83,35 @@ only in a `tag_provenance` field; `image`, `registry_ref`, and base image fields
must use `image@sha256:...`, while a private local artifact may use an exact must use `image@sha256:...`, while a private local artifact may use an exact
`docker_image_id`. `docker_image_id`.
An application canary and the shared deterministic contract prove different
layers. For example, a consumer may instrument one application prompt without
being able to observe the underlying model request. Record that distinction
explicitly; do not infer a model request count from an application prompt
count. The exact-image shared contract supplies the model-request proof.
Likewise, the public image job executes `ruby-rest-sse`; it does not silently
certify Rails persistence, plugin hooks, voice streaming, strict route gates,
or provider hooks. Those appear as `required_consumer_profiles` in the image
matrix and need their own consumer evidence before a full tuple can pass.
The command clears `candidate` after promotion. It sets the repository-wide The command clears `candidate` after promotion. It sets the repository-wide
`migration_state` to `certified` only after every consumer has certified `migration_state` to `certified` only after every consumer has certified
`current` and `previous` tuples and no pending candidate. `current` and `previous` tuples and no pending candidate.
## Bootstrap with a failing baseline
If the observed production baseline fails the current contract, keep its full
coordinates for emergency recovery but mark it failed. Do not create passing
evidence for it and do not promote it into `previous`. The promoter rejects
known-failed baselines even if a document claims `pass`.
The safe bootstrap is two-stage: certify and roll out the new candidate, then
create and canary a distinct consumer rollback commit that preserves the same
known-good client and exact runtime. Only after both immutable consumer commits
have real passing evidence can the manifest honestly contain certified
`current` and `previous` tuples. Until then, `promotion_readiness` remains
blocked and the candidate PR must not be treated as a deploy authorization.
## Rollback ## Rollback
Rollback restores the whole `previous` tuple. Do not roll back only the gem or Rollback restores the whole `previous` tuple. Do not roll back only the gem or

16
evidence/2026-07-18-ajent-rails-alpha7-candidate.json Normal file
View File

@@ -0,0 +1,16 @@
{
"schema_version": 1,
"consumer": "ajent-rails",
"profile": "rails-persisted-turn",
"consumer_commit": "96552893baccd5ccba1592b2585edfef1299b4f9",
"status": "pass",
"certified_at": "2026-07-18T22:03:14Z",
"tuple_sha256": "sha256:2fc1026cf247b5ad5677bfa4673c1073f7c2478769dca0cc8443a0924cebfca3",
"shared_evidence": "evidence/2026-07-18-opencode-ruby-alpha7.json",
"canary": {
"checked_at": "2026-07-18T21:58:00Z",
"docker_image_id": "sha256:239e3dc6cd2958251cda636fb20bd7cd7c46f596bd4c61cb81863ebc49d824e8",
"persisted_final_text": "compat-ok",
"model_request_count": 1
}
}

17
evidence/2026-07-18-mushu-alpha7-candidate.json Normal file
View File

@@ -0,0 +1,17 @@
{
"schema_version": 1,
"consumer": "mushu",
"profile": "ruby-rest-sse",
"consumer_commit": "f26d958b64f214a422aeb629860c8c9a46a4c084",
"status": "pass",
"certified_at": "2026-07-18T22:03:14Z",
"tuple_sha256": "sha256:6171647c7db6d88d52882695c6106a4391e87d0bb1f9e0c48151b2bc1e95b318",
"shared_evidence": "evidence/2026-07-18-opencode-ruby-alpha7.json",
"canary": {
"run_id": "mushu-alpha7-20260718T215300Z-f26d958",
"checked_at": "2026-07-18T21:53:00Z",
"docker_image_id": "sha256:ed6293ca87e6db1dc07870d3ed6da21960743e6e2e904533b3c566d334b58d6a",
"persisted_final_text": "compat-ok",
"model_request_count": 1
}
}

51
evidence/2026-07-18-opencode-ruby-alpha2-regression.json Normal file
View File

@@ -0,0 +1,51 @@
{
"schema_version": 1,
"client": {
"repository": "ajaynomics/opencode-ruby",
"version": "0.0.1.alpha2",
"commit": "889d38332f98f5f7b76d16952b7204d8a5b9a662"
},
"rails_client": {
"repository": "ajaynomics/opencode-rails",
"version": "0.0.1.alpha2",
"commit": "9b0c4cd3cd31bdfebbb89567daca012d68a356ea"
},
"recorded_at": "2026-07-18T22:04:14Z",
"status": "fail",
"failure": {
"expected_text": "compat-ok",
"full_text": "compat-ok\n\ncompat-ok",
"llm_request_count": 1,
"reason": "The client concatenates the streamed and authoritative text, duplicating the final response."
},
"exact_image_contracts": [
{
"image": "ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a",
"server_version": "1.16.1",
"status": "fail"
},
{
"image": "ghcr.io/anomalyco/opencode@sha256:bf9d0e84b7cedef436a8f57db4d48767cd5d8fc6604f666335c1cc916b199a97",
"server_version": "1.17.18",
"status": "fail"
},
{
"image": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e",
"server_version": "1.18.3",
"status": "fail"
},
{
"target": "ajent-production-artifact",
"docker_image_id": "sha256:6fb2b3bf8e4cdf48e16bf6c3475e953df2f8304e77f5efe816ceb46f905eead8",
"server_version": "1.18.3",
"status": "fail"
},
{
"target": "mushu-production-artifact",
"docker_image_id": "sha256:ed6293ca87e6db1dc07870d3ed6da21960743e6e2e904533b3c566d334b58d6a",
"server_version": "0.0.0-permission-shortcuts-202606300721",
"status": "fail"
}
],
"promotion_effect": "The observed alpha2 consumer baselines are not eligible for certified previous tuples. A distinct, tested rollback consumer commit pinned to alpha7 is required before the two-certified-tuples policy can be satisfied."
}

69
evidence/2026-07-18-opencode-ruby-alpha6.json
View File

@@ -1,69 +0,0 @@
{
"schema_version": 1,
"candidate": {
"repository": "ajaynomics/opencode-ruby",
"version": "0.0.1.alpha6",
"commit": "a116b2708cc148f61b1e1fae0405553c099f4202",
"certified_payload_commit": "8adc95985a79b16e6e4bc0c0f827d638f244d509",
"payload_equivalence": "release differs only in .github/workflows/test.yml, which is outside the gem payload",
"transport_implementation_commit": "5113a953db8026697262bd4f97197a3d13077762"
},
"fixture_contract": {
"status": "pass",
"fixture_count": 7
},
"unit_contract": {
"status": "pass",
"runs": 24,
"assertions": 56
},
"image_contracts": [
{
"target": "travelwolf-1.16.1",
"image": "ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a",
"server_version": "1.16.1",
"checked_at": "2026-07-18T20:34:05Z",
"status": "pass"
},
{
"target": "opencode-ajent-1.17.18",
"image": "ghcr.io/anomalyco/opencode@sha256:bf9d0e84b7cedef436a8f57db4d48767cd5d8fc6604f666335c1cc916b199a97",
"server_version": "1.17.18",
"checked_at": "2026-07-18T20:34:08Z",
"status": "pass"
},
{
"target": "ajent-base-1.18.3",
"image": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e",
"server_version": "1.18.3",
"checked_at": "2026-07-18T20:34:08Z",
"status": "pass"
},
{
"target": "ajent-blackline-live-artifact",
"image_id": "sha256:6fb2b3bf8e4cdf48e16bf6c3475e953df2f8304e77f5efe816ceb46f905eead8",
"registry_ref": "docker-registry:5000/opencode-blackline:83eed3c247a352842b08fc5cf339c4d0acdaacba",
"server_version": "1.18.3",
"checked_at": "2026-07-18T20:34:10Z",
"status": "pass"
},
{
"target": "mushu-permission-shortcuts-live-artifact",
"image_id": "sha256:ed6293ca87e6db1dc07870d3ed6da21960743e6e2e904533b3c566d334b58d6a",
"registry_ref": "docker-registry:5000/mushu-opencode:sha-99d6328b1",
"source_commit": "99d6328b18ff8a340a928449a2aa8bc184a063f9",
"server_version": "0.0.0-permission-shortcuts-202606300721",
"checked_at": "2026-07-18T20:34:08Z",
"status": "pass"
}
],
"contract": [
"health",
"create-session",
"subscribe-ready-before-async-prompt",
"deterministic-model-request",
"sse-terminal",
"authoritative-final-exchange",
"delete-session"
]
}

123
evidence/2026-07-18-opencode-ruby-alpha7.json Normal file
View File

@@ -0,0 +1,123 @@
{
"schema_version": 1,
"candidate": {
"opencode_ruby": {
"repository": "ajaynomics/opencode-ruby",
"version": "0.0.1.alpha7",
"commit": "78b6f9c9e9c7d58b699af1c3c17764acd33de798",
"tag_provenance": {
"tag": "v0.0.1.alpha7",
"annotated_tag_object": "abfce3ec8cb1ee468107d0845521769502251645",
"peeled_commit": "78b6f9c9e9c7d58b699af1c3c17764acd33de798"
}
},
"opencode_rails": {
"repository": "ajaynomics/opencode-rails",
"version": "0.0.1.alpha7",
"commit": "2a391ccad1d098e4bb51eb19b6ca52adaa79e5cb",
"tag_provenance": {
"tag": "v0.0.1.alpha7",
"annotated_tag_object": "61e128084b2b665c0ba20b332987e41794948d7c",
"peeled_commit": "2a391ccad1d098e4bb51eb19b6ca52adaa79e5cb"
}
}
},
"fixture_contract": {
"status": "pass",
"fixture_count": 7
},
"opencode_ruby_unit_contract": {
"status": "pass",
"runs": 26,
"assertions": 63
},
"opencode_rails_unit_contract": {
"status": "pass",
"runs": 55,
"assertions": 145
},
"exact_image_contracts": [
{
"target": "travelwolf-1.16.1",
"image": "ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a",
"server_version": "1.16.1",
"checked_at": "2026-07-18T22:03:12Z",
"expected_text": "compat-ok",
"full_text": "compat-ok",
"llm_request_count": 1,
"status": "pass"
},
{
"target": "opencode-ajent-1.17.18",
"image": "ghcr.io/anomalyco/opencode@sha256:bf9d0e84b7cedef436a8f57db4d48767cd5d8fc6604f666335c1cc916b199a97",
"server_version": "1.17.18",
"checked_at": "2026-07-18T22:03:13Z",
"expected_text": "compat-ok",
"full_text": "compat-ok",
"llm_request_count": 1,
"status": "pass"
},
{
"target": "ajent-base-1.18.3",
"image": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e",
"server_version": "1.18.3",
"checked_at": "2026-07-18T22:03:14Z",
"expected_text": "compat-ok",
"full_text": "compat-ok",
"llm_request_count": 1,
"status": "pass"
}
],
"consumer_canaries": [
{
"consumer": "ajent-rails",
"consumer_commit": "96552893baccd5ccba1592b2585edfef1299b4f9",
"tag_provenance": "ajent-opencode-blackline-canary:96552893",
"docker_image_id": "sha256:239e3dc6cd2958251cda636fb20bd7cd7c46f596bd4c61cb81863ebc49d824e8",
"source_commit": "96552893baccd5ccba1592b2585edfef1299b4f9",
"upstream_base": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e",
"server_version": "1.18.3",
"checked_at": "2026-07-18T21:58:00Z",
"application_path": "Blackline::GenerateResponseJob",
"persisted_final_text": "compat-ok",
"model_request_count": 1,
"status": "pass"
},
{
"consumer": "mushu",
"consumer_commit": "f26d958b64f214a422aeb629860c8c9a46a4c084",
"run_id": "mushu-alpha7-20260718T215300Z-f26d958",
"tag_provenance": "docker-registry:5000/mushu-opencode:sha-99d6328b1",
"docker_image_id": "sha256:ed6293ca87e6db1dc07870d3ed6da21960743e6e2e904533b3c566d334b58d6a",
"source_commit": "99d6328b18ff8a340a928449a2aa8bc184a063f9",
"server_version": "0.0.0-permission-shortcuts-202606300721",
"checked_at": "2026-07-18T21:53:00Z",
"persisted_final_text": "compat-ok",
"model_request_count": 1,
"status": "pass"
},
{
"consumer": "travelwolf",
"consumer_commit": "19747b59fa21e12560afd9edd73ed674ab3a644c",
"run_id": "20260718215135-8d8102c0",
"image": "ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a",
"server_version": "1.16.1",
"started_at": "2026-07-18T21:51:35Z",
"checked_at": "2026-07-18T21:51:56.552Z",
"final_text": "TRAVELWOLF_ALPHA7_SUBSCRIBE_READY",
"application_prompt_count": 1,
"model_request_count_observed": false,
"cleanup_status": "pass",
"status": "pass"
}
],
"contract": [
"health",
"create-session",
"subscribe-ready-before-async-prompt",
"exactly-one-deterministic-model-request",
"sse-terminal",
"exact-authoritative-final-exchange",
"delete-session"
]
}

18
evidence/2026-07-18-travelwolf-alpha7-candidate.json Normal file
View File

@@ -0,0 +1,18 @@
{
"schema_version": 1,
"consumer": "travelwolf",
"profile": "rails-persisted-turn",
"consumer_commit": "19747b59fa21e12560afd9edd73ed674ab3a644c",
"status": "pass",
"certified_at": "2026-07-18T22:03:14Z",
"tuple_sha256": "sha256:a7519e661f74d86788928540429c953edcbf20d26321c0dd2b72cd0a1b995b61",
"shared_evidence": "evidence/2026-07-18-opencode-ruby-alpha7.json",
"canary": {
"run_id": "20260718215135-8d8102c0",
"checked_at": "2026-07-18T21:51:56.552Z",
"final_text": "TRAVELWOLF_ALPHA7_SUBSCRIBE_READY",
"application_prompt_count": 1,
"model_request_count_observed": false,
"cleanup_status": "pass"
}
}

6
lib/opencode_compat/runtime_tuple_promoter.rb
View File

@@ -15,6 +15,7 @@ module OpenCodeCompat
IMMUTABLE_IMAGE = /\A[^@\s]+@sha256:[0-9a-f]{64}\z/ IMMUTABLE_IMAGE = /\A[^@\s]+@sha256:[0-9a-f]{64}\z/
UTC_TIMESTAMP = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?Z\z/ UTC_TIMESTAMP = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?Z\z/
CERTIFICATION_STATUS = "pass" CERTIFICATION_STATUS = "pass"
NON_CERTIFIABLE_TUPLE_STATUSES = %w[observed-production-contract-failed].freeze
TUPLE_METADATA_KEYS = %w[ TUPLE_METADATA_KEYS = %w[
certification certification
certified_at certified_at
@@ -172,6 +173,11 @@ module OpenCodeCompat
end end
def certify_previous!(tuple, consumer:, profile:, supplied:, expected_fingerprint:) def certify_previous!(tuple, consumer:, profile:, supplied:, expected_fingerprint:)
if NON_CERTIFIABLE_TUPLE_STATUSES.include?(tuple["status"])
raise PromotionError,
"current tuple is known to fail the compatibility contract and cannot become a certified rollback"
end
if tuple["status"] == "certified" if tuple["status"] == "certified"
validate_recorded_certification!( validate_recorded_certification!(
tuple, tuple,

9
manifests/client-candidate.json
View File

@@ -1,6 +1,11 @@
{ {
"repository": "ajaynomics/opencode-ruby", "repository": "ajaynomics/opencode-ruby",
"ref": "a116b2708cc148f61b1e1fae0405553c099f4202", "ref": "78b6f9c9e9c7d58b699af1c3c17764acd33de798",
"version": "0.0.1.alpha6", "version": "0.0.1.alpha7",
"tag_provenance": {
"tag": "v0.0.1.alpha7",
"annotated_tag_object": "abfce3ec8cb1ee468107d0845521769502251645",
"peeled_commit": "78b6f9c9e9c7d58b699af1c3c17764acd33de798"
},
"status": "candidate" "status": "candidate"
} }

61
manifests/image-matrix.json
View File

@@ -7,8 +7,14 @@
"image": "ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a", "image": "ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a",
"tag_provenance": "ghcr.io/anomalyco/opencode:1.16.1", "tag_provenance": "ghcr.io/anomalyco/opencode:1.16.1",
"consumers": ["travelwolf"], "consumers": ["travelwolf"],
"profiles": ["ruby-rest-sse", "rails-persisted-turn"], "profiles": ["ruby-rest-sse"],
"certification_status": "certified" "required_consumer_profiles": ["rails-persisted-turn"],
"certification_status": "certified",
"certified_client_commit": "78b6f9c9e9c7d58b699af1c3c17764acd33de798",
"certified_at": "2026-07-18T22:03:12Z",
"expected_text": "compat-ok",
"full_text": "compat-ok",
"llm_request_count": 1
}, },
{ {
"id": "upstream-1.17.18", "id": "upstream-1.17.18",
@@ -16,8 +22,14 @@
"image": "ghcr.io/anomalyco/opencode@sha256:bf9d0e84b7cedef436a8f57db4d48767cd5d8fc6604f666335c1cc916b199a97", "image": "ghcr.io/anomalyco/opencode@sha256:bf9d0e84b7cedef436a8f57db4d48767cd5d8fc6604f666335c1cc916b199a97",
"tag_provenance": "ghcr.io/anomalyco/opencode:1.17.18", "tag_provenance": "ghcr.io/anomalyco/opencode:1.17.18",
"consumers": ["opencode-ajent"], "consumers": ["opencode-ajent"],
"profiles": ["plugin-ledger", "ruby-rest-sse"], "profiles": ["ruby-rest-sse"],
"certification_status": "certified" "required_consumer_profiles": ["plugin-ledger"],
"certification_status": "certified",
"certified_client_commit": "78b6f9c9e9c7d58b699af1c3c17764acd33de798",
"certified_at": "2026-07-18T22:03:13Z",
"expected_text": "compat-ok",
"full_text": "compat-ok",
"llm_request_count": 1
}, },
{ {
"id": "upstream-1.18.3", "id": "upstream-1.18.3",
@@ -25,34 +37,49 @@
"image": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e", "image": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e",
"tag_provenance": "ghcr.io/anomalyco/opencode:1.18.3", "tag_provenance": "ghcr.io/anomalyco/opencode:1.18.3",
"consumers": ["ajent-rails"], "consumers": ["ajent-rails"],
"profiles": ["ruby-rest-sse", "rails-persisted-turn", "plugin-ledger"], "profiles": ["ruby-rest-sse"],
"certification_status": "certified" "required_consumer_profiles": ["rails-persisted-turn", "plugin-ledger"],
"certification_status": "certified",
"certified_client_commit": "78b6f9c9e9c7d58b699af1c3c17764acd33de798",
"certified_at": "2026-07-18T22:03:14Z",
"expected_text": "compat-ok",
"full_text": "compat-ok",
"llm_request_count": 1
} }
], ],
"host_canary": [ "host_canary": [
{ {
"id": "ajent-blackline-live", "id": "ajent-blackline-alpha7-canary",
"consumer": "ajent-rails", "consumer": "ajent-rails",
"registry_ref": "docker-registry:5000/opencode-blackline:83eed3c247a352842b08fc5cf339c4d0acdaacba", "consumer_commit": "96552893baccd5ccba1592b2585edfef1299b4f9",
"docker_image_id": "sha256:6fb2b3bf8e4cdf48e16bf6c3475e953df2f8304e77f5efe816ceb46f905eead8", "tag_provenance": "ajent-opencode-blackline-canary:96552893",
"docker_image_id": "sha256:239e3dc6cd2958251cda636fb20bd7cd7c46f596bd4c61cb81863ebc49d824e8",
"source_commit": "96552893baccd5ccba1592b2585edfef1299b4f9",
"reported_version": "1.18.3", "reported_version": "1.18.3",
"upstream_base": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e", "upstream_base": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e",
"profiles": ["ruby-rest-sse", "rails-persisted-turn", "plugin-ledger"], "profiles": ["rails-persisted-turn"],
"certification_status": "candidate-certified", "certification_status": "candidate-certified",
"certified_candidate": "a116b2708cc148f61b1e1fae0405553c099f4202", "certified_client_commit": "78b6f9c9e9c7d58b699af1c3c17764acd33de798",
"certified_at": "2026-07-18T20:34:10Z" "certified_rails_commit": "2a391ccad1d098e4bb51eb19b6ca52adaa79e5cb",
"certified_at": "2026-07-18T21:58:00Z",
"persisted_final_text": "compat-ok",
"model_request_count": 1
}, },
{ {
"id": "mushu-permission-shortcuts-live", "id": "mushu-permission-shortcuts-alpha7-canary",
"consumer": "mushu", "consumer": "mushu",
"registry_ref": "docker-registry:5000/mushu-opencode:sha-99d6328b1", "consumer_commit": "f26d958b64f214a422aeb629860c8c9a46a4c084",
"run_id": "mushu-alpha7-20260718T215300Z-f26d958",
"tag_provenance": "docker-registry:5000/mushu-opencode:sha-99d6328b1",
"docker_image_id": "sha256:ed6293ca87e6db1dc07870d3ed6da21960743e6e2e904533b3c566d334b58d6a", "docker_image_id": "sha256:ed6293ca87e6db1dc07870d3ed6da21960743e6e2e904533b3c566d334b58d6a",
"reported_version": "0.0.0-permission-shortcuts-202606300721",
"source_commit": "99d6328b18ff8a340a928449a2aa8bc184a063f9", "source_commit": "99d6328b18ff8a340a928449a2aa8bc184a063f9",
"reported_version": "0.0.0-permission-shortcuts-202606300721",
"profiles": ["ruby-rest-sse"], "profiles": ["ruby-rest-sse"],
"certification_status": "candidate-certified", "certification_status": "candidate-certified",
"certified_candidate": "a116b2708cc148f61b1e1fae0405553c099f4202", "certified_client_commit": "78b6f9c9e9c7d58b699af1c3c17764acd33de798",
"certified_at": "2026-07-18T20:34:08Z" "certified_at": "2026-07-18T21:53:00Z",
"persisted_final_text": "compat-ok",
"model_request_count": 1
} }
] ]
} }

123
manifests/runtime-tuples.json
View File

@@ -1,16 +1,28 @@
{ {
"schema_version": 1, "schema_version": 1,
"migration_state": "candidate", "migration_state": "candidate",
"promotion_readiness": {
"status": "blocked",
"reason": "Every observed alpha2 baseline fails the hardened exact-final-text contract, so none can be certified as the previous rollback tuple.",
"evidence": "evidence/2026-07-18-opencode-ruby-alpha2-regression.json",
"required_action": "Create and canary a distinct rollback consumer commit that pins alpha7 and the exact runtime before promoting any candidate."
},
"consumers": { "consumers": {
"ajent-rails": { "ajent-rails": {
"profile": "rails-persisted-turn", "profile": "rails-persisted-turn",
"current": { "current": {
"status": "observed-production", "status": "observed-production-contract-failed",
"consumer_commit": "83eed3c247a352842b08fc5cf339c4d0acdaacba", "consumer_commit": "83eed3c247a352842b08fc5cf339c4d0acdaacba",
"opencode_ruby": {"version": "0.0.1.alpha2", "git_commit": "889d383"}, "opencode_ruby": {
"opencode_rails": {"version": "0.0.1.alpha2", "git_commit": "9b0c4cd"}, "version": "0.0.1.alpha2",
"git_commit": "889d38332f98f5f7b76d16952b7204d8a5b9a662"
},
"opencode_rails": {
"version": "0.0.1.alpha2",
"git_commit": "9b0c4cd3cd31bdfebbb89567daca012d68a356ea"
},
"runtime": { "runtime": {
"registry_ref": "docker-registry:5000/opencode-blackline:83eed3c247a352842b08fc5cf339c4d0acdaacba", "tag_provenance": "docker-registry:5000/opencode-blackline:83eed3c247a352842b08fc5cf339c4d0acdaacba",
"docker_image_id": "sha256:6fb2b3bf8e4cdf48e16bf6c3475e953df2f8304e77f5efe816ceb46f905eead8", "docker_image_id": "sha256:6fb2b3bf8e4cdf48e16bf6c3475e953df2f8304e77f5efe816ceb46f905eead8",
"reported_version": "1.18.3", "reported_version": "1.18.3",
"upstream_base": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e" "upstream_base": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e"
@@ -18,24 +30,54 @@
}, },
"candidate": { "candidate": {
"status": "compatibility-certified", "status": "compatibility-certified",
"opencode_ruby": {"version": "0.0.1.alpha6", "git_commit": "a116b2708cc148f61b1e1fae0405553c099f4202"}, "consumer_commit": "96552893baccd5ccba1592b2585edfef1299b4f9",
"opencode_rails": {"version": "0.0.1.alpha6", "git_commit": "17025f0ed955899c57830eaea34da49f07250fcc"}, "opencode_ruby": {
"version": "0.0.1.alpha7",
"git_commit": "78b6f9c9e9c7d58b699af1c3c17764acd33de798",
"tag_provenance": {
"tag": "v0.0.1.alpha7",
"annotated_tag_object": "abfce3ec8cb1ee468107d0845521769502251645"
}
},
"opencode_rails": {
"version": "0.0.1.alpha7",
"git_commit": "2a391ccad1d098e4bb51eb19b6ca52adaa79e5cb",
"tag_provenance": {
"tag": "v0.0.1.alpha7",
"annotated_tag_object": "61e128084b2b665c0ba20b332987e41794948d7c"
}
},
"runtime": { "runtime": {
"docker_image_id": "sha256:6fb2b3bf8e4cdf48e16bf6c3475e953df2f8304e77f5efe816ceb46f905eead8", "tag_provenance": "ajent-opencode-blackline-canary:96552893",
"docker_image_id": "sha256:239e3dc6cd2958251cda636fb20bd7cd7c46f596bd4c61cb81863ebc49d824e8",
"source_commit": "96552893baccd5ccba1592b2585edfef1299b4f9",
"reported_version": "1.18.3", "reported_version": "1.18.3",
"upstream_base": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e" "upstream_base": "ghcr.io/anomalyco/opencode@sha256:c2d5d6398df72aac85cb1bdc8f900c71a9b75a33fb7c0a76dc1484e4b126e41e"
}, },
"certified_at": "2026-07-18T20:34:10Z" "application_canary": {
"checked_at": "2026-07-18T21:58:00Z",
"path": "Blackline::GenerateResponseJob",
"persisted_final_text": "compat-ok",
"model_request_count": 1,
"status": "pass"
},
"certified_at": "2026-07-18T22:03:14Z"
}, },
"previous": null "previous": null
}, },
"travelwolf": { "travelwolf": {
"profile": "rails-persisted-turn", "profile": "rails-persisted-turn",
"current": { "current": {
"status": "repository-configured", "status": "observed-production-contract-failed",
"consumer_commit": "1ca756d64f058c9e0379f69fc4a3d53330cfcd15", "consumer_commit": "1ca756d64f058c9e0379f69fc4a3d53330cfcd15",
"opencode_ruby": {"version": "0.0.1.alpha2", "git_commit": "889d383"}, "opencode_ruby": {
"opencode_rails": {"version": "0.0.1.alpha2", "git_commit": "9b0c4cd"}, "version": "0.0.1.alpha2",
"git_commit": "889d38332f98f5f7b76d16952b7204d8a5b9a662"
},
"opencode_rails": {
"version": "0.0.1.alpha2",
"git_commit": "9b0c4cd3cd31bdfebbb89567daca012d68a356ea"
},
"runtime": { "runtime": {
"image": "ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a", "image": "ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a",
"reported_version": "1.16.1" "reported_version": "1.16.1"
@@ -43,24 +85,51 @@
}, },
"candidate": { "candidate": {
"status": "compatibility-certified", "status": "compatibility-certified",
"opencode_ruby": {"version": "0.0.1.alpha6", "git_commit": "a116b2708cc148f61b1e1fae0405553c099f4202"}, "consumer_commit": "19747b59fa21e12560afd9edd73ed674ab3a644c",
"opencode_rails": {"version": "0.0.1.alpha6", "git_commit": "17025f0ed955899c57830eaea34da49f07250fcc"}, "opencode_ruby": {
"version": "0.0.1.alpha7",
"git_commit": "78b6f9c9e9c7d58b699af1c3c17764acd33de798",
"tag_provenance": {
"tag": "v0.0.1.alpha7",
"annotated_tag_object": "abfce3ec8cb1ee468107d0845521769502251645"
}
},
"opencode_rails": {
"version": "0.0.1.alpha7",
"git_commit": "2a391ccad1d098e4bb51eb19b6ca52adaa79e5cb",
"tag_provenance": {
"tag": "v0.0.1.alpha7",
"annotated_tag_object": "61e128084b2b665c0ba20b332987e41794948d7c"
}
},
"runtime": { "runtime": {
"image": "ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a", "image": "ghcr.io/anomalyco/opencode@sha256:e975a0647576016dfdf77d54b979ca30d32b4750472c10263e9894aad6628c2a",
"reported_version": "1.16.1" "reported_version": "1.16.1"
}, },
"certified_at": "2026-07-18T20:34:05Z" "application_canary": {
"run_id": "20260718215135-8d8102c0",
"checked_at": "2026-07-18T21:51:56.552Z",
"final_text": "TRAVELWOLF_ALPHA7_SUBSCRIBE_READY",
"application_prompt_count": 1,
"model_request_count_observed": false,
"cleanup_status": "pass",
"status": "pass"
},
"certified_at": "2026-07-18T22:03:14Z"
}, },
"previous": null "previous": null
}, },
"mushu": { "mushu": {
"profile": "ruby-rest-sse", "profile": "ruby-rest-sse",
"current": { "current": {
"status": "observed-production", "status": "observed-production-contract-failed",
"consumer_commit": "fe16e5bab028e3c92cd5508ef2cebd9bfa23386f", "consumer_commit": "fe16e5bab028e3c92cd5508ef2cebd9bfa23386f",
"opencode_ruby": {"version": "0.0.1.alpha2", "git_commit": "889d383"}, "opencode_ruby": {
"version": "0.0.1.alpha2",
"git_commit": "889d38332f98f5f7b76d16952b7204d8a5b9a662"
},
"runtime": { "runtime": {
"registry_ref": "docker-registry:5000/mushu-opencode:sha-99d6328b1", "tag_provenance": "docker-registry:5000/mushu-opencode:sha-99d6328b1",
"docker_image_id": "sha256:ed6293ca87e6db1dc07870d3ed6da21960743e6e2e904533b3c566d334b58d6a", "docker_image_id": "sha256:ed6293ca87e6db1dc07870d3ed6da21960743e6e2e904533b3c566d334b58d6a",
"source_commit": "99d6328b18ff8a340a928449a2aa8bc184a063f9", "source_commit": "99d6328b18ff8a340a928449a2aa8bc184a063f9",
"reported_version": "0.0.0-permission-shortcuts-202606300721" "reported_version": "0.0.0-permission-shortcuts-202606300721"
@@ -68,13 +137,29 @@
}, },
"candidate": { "candidate": {
"status": "compatibility-certified", "status": "compatibility-certified",
"opencode_ruby": {"version": "0.0.1.alpha6", "git_commit": "a116b2708cc148f61b1e1fae0405553c099f4202"}, "consumer_commit": "f26d958b64f214a422aeb629860c8c9a46a4c084",
"opencode_ruby": {
"version": "0.0.1.alpha7",
"git_commit": "78b6f9c9e9c7d58b699af1c3c17764acd33de798",
"tag_provenance": {
"tag": "v0.0.1.alpha7",
"annotated_tag_object": "abfce3ec8cb1ee468107d0845521769502251645"
}
},
"runtime": { "runtime": {
"tag_provenance": "docker-registry:5000/mushu-opencode:sha-99d6328b1",
"docker_image_id": "sha256:ed6293ca87e6db1dc07870d3ed6da21960743e6e2e904533b3c566d334b58d6a", "docker_image_id": "sha256:ed6293ca87e6db1dc07870d3ed6da21960743e6e2e904533b3c566d334b58d6a",
"source_commit": "99d6328b18ff8a340a928449a2aa8bc184a063f9", "source_commit": "99d6328b18ff8a340a928449a2aa8bc184a063f9",
"reported_version": "0.0.0-permission-shortcuts-202606300721" "reported_version": "0.0.0-permission-shortcuts-202606300721"
}, },
"certified_at": "2026-07-18T20:34:08Z" "application_canary": {
"run_id": "mushu-alpha7-20260718T215300Z-f26d958",
"checked_at": "2026-07-18T21:53:00Z",
"persisted_final_text": "compat-ok",
"model_request_count": 1,
"status": "pass"
},
"certified_at": "2026-07-18T22:03:14Z"
}, },
"previous": null "previous": null
} }

33
ruby/exact_live_contract.rb Normal file
View File

@@ -0,0 +1,33 @@
# frozen_string_literal: true
module OpenCodeCompat
module ExactLiveContract
class ContractError < StandardError; end
module_function
def assert_final_text!(actual, expected)
return if actual == expected
raise ContractError, "Expected final text to equal #{expected.inspect}, got #{actual.inspect}"
end
def assert_model_request_count!(value)
count = Integer(value, 10)
return if count == 1
raise ContractError, "Expected exactly one deterministic model request, got #{count}"
rescue ArgumentError, TypeError
raise ContractError, "Deterministic model request count is not an integer: #{value.inspect}"
end
end
end
if $PROGRAM_NAME == __FILE__
begin
OpenCodeCompat::ExactLiveContract.assert_model_request_count!(ARGV.fetch(0))
rescue IndexError, OpenCodeCompat::ExactLiveContract::ContractError => error
warn error.message
exit 1
end
end

5
ruby/live_probe.rb
View File

@@ -2,6 +2,7 @@
require "json" require "json"
require "time" require "time"
require_relative "exact_live_contract"
gem_path = File.expand_path(ENV.fetch("OPENCODE_RUBY_PATH")) gem_path = File.expand_path(ENV.fetch("OPENCODE_RUBY_PATH"))
$LOAD_PATH.unshift(File.join(gem_path, "lib")) $LOAD_PATH.unshift(File.join(gem_path, "lib"))
@@ -32,9 +33,7 @@ begin
observed_parts << JSON.parse(JSON.generate(part)) observed_parts << JSON.parse(JSON.generate(part))
end end
unless result.full_text.include?(expected_text) OpenCodeCompat::ExactLiveContract.assert_final_text!(result.full_text, expected_text)
raise "Expected final text to include #{expected_text.inspect}, got #{result.full_text.inspect}"
end
messages = client.get_messages(session_id) messages = client.get_messages(session_id)
assistant_messages = Array(messages).select { |message| message.dig(:info, :role) == "assistant" } assistant_messages = Array(messages).select { |message| message.dig(:info, :role) == "assistant" }

3
scripts/record_upstream_candidate.rb
View File

@@ -34,7 +34,8 @@ unless matrix.fetch("public_ci").any? { |target| target.fetch("image") == image
"image" => image, "image" => image,
"tag_provenance" => "ghcr.io/anomalyco/opencode:#{version}", "tag_provenance" => "ghcr.io/anomalyco/opencode:#{version}",
"consumers" => ["upstream-candidate"], "consumers" => ["upstream-candidate"],
"profiles" => ["ruby-rest-sse", "rails-persisted-turn", "voice-stream", "strict-v2", "plugin-ledger", "provider-hooks"], "profiles" => ["ruby-rest-sse"],
"required_consumer_profiles" => ["rails-persisted-turn", "voice-stream", "strict-v2", "plugin-ledger", "provider-hooks"],
"certification_status" => "pending" "certification_status" => "pending"
} }
end end

5
scripts/run_image_contract.sh
View File

@@ -127,10 +127,7 @@ OPENCODE_RUBY_COMMIT="$gem_commit" \
llm_stats="$(docker exec "$llm_container_name" wget -qO- http://127.0.0.1:8080/stats)" llm_stats="$(docker exec "$llm_container_name" wget -qO- http://127.0.0.1:8080/stats)"
request_count="$(jq -r '.request_count' <<<"$llm_stats")" request_count="$(jq -r '.request_count' <<<"$llm_stats")"
if [[ "$request_count" -lt 1 ]]; then ruby "$repo_root/ruby/exact_live_contract.rb" "$request_count"
echo "OpenCode never called the deterministic model" >&2
exit 1
fi
jq -cn \ jq -cn \
--arg status pass \ --arg status pass \

29
test/exact_live_contract_test.rb Normal file
View File

@@ -0,0 +1,29 @@
# frozen_string_literal: true
require "minitest/autorun"
require_relative "../ruby/exact_live_contract"
class ExactLiveContractTest < Minitest::Test
def test_accepts_only_exact_final_text
assert_nil OpenCodeCompat::ExactLiveContract.assert_final_text!("compat-ok", "compat-ok")
error = assert_raises(OpenCodeCompat::ExactLiveContract::ContractError) do
OpenCodeCompat::ExactLiveContract.assert_final_text!("compat-ok\n\ncompat-ok", "compat-ok")
end
assert_match(/equal/, error.message)
assert_raises(OpenCodeCompat::ExactLiveContract::ContractError) do
OpenCodeCompat::ExactLiveContract.assert_final_text!("prefix compat-ok", "compat-ok")
end
end
def test_accepts_only_one_model_request
assert_nil OpenCodeCompat::ExactLiveContract.assert_model_request_count!("1")
%w[0 2 not-a-number].each do |count|
assert_raises(OpenCodeCompat::ExactLiveContract::ContractError) do
OpenCodeCompat::ExactLiveContract.assert_model_request_count!(count)
end
end
end
end

84
test/repository_test.rb
View File

@@ -2,6 +2,7 @@
require "json" require "json"
require "minitest/autorun" require "minitest/autorun"
require_relative "../lib/opencode_compat/runtime_tuple_promoter"
class RepositoryTest < Minitest::Test class RepositoryTest < Minitest::Test
ROOT = File.expand_path("..", __dir__) ROOT = File.expand_path("..", __dir__)
@@ -42,9 +43,24 @@ class RepositoryTest < Minitest::Test
assert_match %r{\Aghcr\.io/anomalyco/opencode@sha256:[0-9a-f]{64}\z}, target.fetch("image") assert_match %r{\Aghcr\.io/anomalyco/opencode@sha256:[0-9a-f]{64}\z}, target.fetch("image")
refute_includes target.fetch("image"), ":latest" refute_includes target.fetch("image"), ":latest"
refute_empty target.fetch("profiles") refute_empty target.fetch("profiles")
next unless target.fetch("certification_status") == "certified"
assert_match(/\A[0-9a-f]{40}\z/, target.fetch("certified_client_commit"))
assert_equal target.fetch("expected_text"), target.fetch("full_text")
assert_equal 1, target.fetch("llm_request_count")
end end
end end
def test_candidate_is_bound_to_an_annotated_tag
candidate = json("manifests/client-candidate.json")
provenance = candidate.fetch("tag_provenance")
assert_match(/\A[0-9a-f]{40}\z/, candidate.fetch("ref"))
assert_match(/\Av\d+\.\d+\.\d+\.alpha\d+\z/, provenance.fetch("tag"))
assert_match(/\A[0-9a-f]{40}\z/, provenance.fetch("annotated_tag_object"))
assert_equal candidate.fetch("ref"), provenance.fetch("peeled_commit")
end
def test_certified_migration_keeps_previous_tuple def test_certified_migration_keeps_previous_tuple
tuples = json("manifests/runtime-tuples.json") tuples = json("manifests/runtime-tuples.json")
return unless tuples.fetch("migration_state") == "certified" return unless tuples.fetch("migration_state") == "certified"
@@ -68,6 +84,64 @@ class RepositoryTest < Minitest::Test
end end
end end
def test_runtime_tuples_use_full_commits_and_no_mutable_registry_coordinate
tuples = json("manifests/runtime-tuples.json")
tuples.fetch("consumers").each_value do |consumer|
%w[current candidate previous].each do |slot|
tuple = consumer[slot]
next unless tuple
assert_match(/\A[0-9a-f]{40}\z/, tuple.fetch("consumer_commit"))
%w[opencode_ruby opencode_rails].each do |client_name|
client = tuple[client_name]
assert_match(/\A[0-9a-f]{40}\z/, client.fetch("git_commit")) if client
end
runtime = tuple.fetch("runtime")
if runtime.key?("registry_ref")
assert_match(/\A[^@\s]+@sha256:[0-9a-f]{64}\z/, runtime.fetch("registry_ref"))
end
if runtime.key?("tag_provenance")
refute_empty runtime.fetch("tag_provenance")
end
end
end
end
def test_failed_alpha2_baselines_block_promotion_instead_of_becoming_previous
tuples = json("manifests/runtime-tuples.json")
readiness = tuples.fetch("promotion_readiness")
assert_equal "blocked", readiness.fetch("status")
assert_path_exists File.join(ROOT, readiness.fetch("evidence"))
tuples.fetch("consumers").each_value do |consumer|
assert_equal "observed-production-contract-failed", consumer.dig("current", "status")
assert_nil consumer.fetch("previous")
end
end
def test_candidate_evidence_is_bound_to_complete_tuple_fingerprints
promoter = OpenCodeCompat::RuntimeTuplePromoter.new(root: ROOT)
evidence_paths = {
"ajent-rails" => "evidence/2026-07-18-ajent-rails-alpha7-candidate.json",
"travelwolf" => "evidence/2026-07-18-travelwolf-alpha7-candidate.json",
"mushu" => "evidence/2026-07-18-mushu-alpha7-candidate.json"
}
evidence_paths.each do |consumer, path|
evidence = json(path)
fingerprints = promoter.fingerprints(
consumer: consumer,
consumer_commit: evidence.fetch("consumer_commit")
)
assert_equal consumer, evidence.fetch("consumer")
assert_equal fingerprints.fetch("profile"), evidence.fetch("profile")
assert_equal fingerprints.fetch("candidate_tuple_sha256"), evidence.fetch("tuple_sha256")
assert_equal "pass", evidence.fetch("status")
end
end
def test_watcher_has_no_deployment_commands def test_watcher_has_no_deployment_commands
workflow = File.read(File.join(ROOT, ".github/workflows/watch-upstream.yml")) workflow = File.read(File.join(ROOT, ".github/workflows/watch-upstream.yml"))
refute_match(/\b(kamal|kubectl|helm|nomad|docker\s+service)\b/i, workflow) refute_match(/\b(kamal|kubectl|helm|nomad|docker\s+service)\b/i, workflow)
@@ -84,4 +158,14 @@ class RepositoryTest < Minitest::Test
refute_match(/`[^`]+`/, implementation) refute_match(/`[^`]+`/, implementation)
refute_match(/\b(kamal|kubectl|helm|nomad|docker\s+service)\b/i, implementation) refute_match(/\b(kamal|kubectl|helm|nomad|docker\s+service)\b/i, implementation)
end end
def test_live_contract_requires_exact_text_and_one_model_request
probe = File.read(File.join(ROOT, "ruby/live_probe.rb"))
runner = File.read(File.join(ROOT, "scripts/run_image_contract.sh"))
assert_includes probe, "ExactLiveContract.assert_final_text!"
refute_includes probe, "full_text.include?"
assert_includes runner, "exact_live_contract.rb"
refute_match(/request_count.*-lt\s+1/, runner)
end
end end

24
test/runtime_tuple_promoter_test.rb
View File

@@ -155,6 +155,26 @@ class RuntimeTuplePromoterTest < Minitest::Test
assert_equal before, File.binread(manifest_path) assert_equal before, File.binread(manifest_path)
end end
def test_rejects_known_failed_baseline_even_with_passing_evidence
manifest = valid_manifest
manifest.dig("consumers", CONSUMER, "current")["status"] = "observed-production-contract-failed"
write_manifest(manifest)
candidate, previous = write_matching_evidence
before = File.binread(manifest_path)
error = assert_raises(OpenCodeCompat::PromotionError) do
@promoter.promote(
consumer: CONSUMER,
consumer_commit: CANDIDATE_COMMIT,
certification: certification(CANDIDATE_TIME, candidate),
previous_certification: certification(CURRENT_TIME, previous)
)
end
assert_match(/known to fail/, error.message)
assert_equal before, File.binread(manifest_path)
end
def test_rejects_evidence_that_does_not_match_the_complete_tuple def test_rejects_evidence_that_does_not_match_the_complete_tuple
candidate, previous = write_matching_evidence candidate, previous = write_matching_evidence
manifest = JSON.parse(File.read(manifest_path)) manifest = JSON.parse(File.read(manifest_path))
@@ -249,8 +269,8 @@ class RuntimeTuplePromoterTest < Minitest::Test
"candidate" => { "candidate" => {
"status" => "compatibility-certified", "status" => "compatibility-certified",
"certified_at" => "2026-07-18T10:00:00Z", "certified_at" => "2026-07-18T10:00:00Z",
"opencode_ruby" => {"version" => "0.0.1.alpha6", "git_commit" => RUBY_CANDIDATE}, "opencode_ruby" => {"version" => "0.0.1.alpha7", "git_commit" => RUBY_CANDIDATE},
"opencode_rails" => {"version" => "0.0.1.alpha6", "git_commit" => RAILS_CANDIDATE}, "opencode_rails" => {"version" => "0.0.1.alpha7", "git_commit" => RAILS_CANDIDATE},
"runtime" => {"image" => CANDIDATE_IMAGE, "reported_version" => "1.18.3"} "runtime" => {"image" => CANDIDATE_IMAGE, "reported_version" => "1.18.3"}
}, },
"previous" => nil "previous" => nil