01
Nondeterministic output
Equivalent prompts can produce materially different plans, patches, and explanations.
Independent engineering case study
A governed, human-in-the-loop control plane for AI-assisted software delivery.
An inspectable workflow that constrains agent actions and context, requires human decisions, and binds evidence to the tested change.
Seven-minute guided walkthrough
Start with the thesis, drill into one synthetic issue, make a distinct human decision, and finish at the recorded sandbox evidence and production boundaries.
The problem
A coding agent can produce a plausible patch while the surrounding process remains unsafe, stale, or impossible to audit. The workbench treats those control failures as first-class product problems.
01
Equivalent prompts can produce materially different plans, patches, and explanations.
02
Old decisions and unbounded repository data can quietly distort the next stage.
03
Broad credentials collapse suggestion, execution, and authorization into one action.
04
Without inputs, versions, and digests, an artifact cannot explain how it came to be.
05
Evidence that is not bound to the tested commit can outlive the code it actually covered.
06
An AI system must not be the sole authority that accepts its own risky work.
Design principles
System workflow
A stage advances only when its required input, review state, and policy gates agree. A revised upstream artifact marks dependent work stale instead of pretending the old result remains valid.
Architecture
Four responsibility planes keep workflow policy, runtime activity, selected context, and validation evidence independently inspectable.
The Control Plane owns workflow state, agent and tool policy, approvals, and budgets. It authorizes the Execution Plane to run a stage with bounded tools; selects a manifest from the Context Plane; and requires the Validation Plane to attach check results and human decisions to the change. Execution does not approve its own output.
Illustrative adapter set
Jira, GitHub, Angular, .NET, and Oracle are one synthetic reference stack behind vendor-neutral issue, repository, UI, service, and database boundaries. They are not a required environment and no external adapter is contacted by the demo.
Functional versus simulated
The matrix distinguishes repository behavior from synthetic representations and future work.
| Capability | Classification | Evidence boundary |
|---|---|---|
| Case-study navigation and demo routing | Functional | Static routes and validated browser-local deep links for screens, synthetic issues, artifacts, Settings subviews, scenarios, and the guided walkthrough. |
| Workflow transitions, retry, redo, and stale behavior | Functional | Typed reducer state, confirmed deterministic reset, and synthetic fixtures. |
| Artifact, pull-request, and validation views | Functional UI | Viewing and local review state work; all displayed records and metrics are synthetic. |
| Versioned agent/tool registry and capability cards | Functional | JSON Schema validation, SHA-256 content hashes, approved-version gating, manifest invalidation, search/detail UI, and generated public JSON are exercised in this repository. |
| Repository-owned local MCP toy-repository slice | Functional locally | A repository command discovers and invokes bounded tools over stdio in a disposable synthetic repository, then emits sanitized static evidence. The browser never connects to the process. |
| Explicit local Docker validation slice | Functional locally | A developer command applies one deterministic allow-listed patch to a temporary synthetic repository, runs fixed build/tests with container networking disabled, and emits validated recorded evidence. The public site cannot invoke it. |
| Optional E2B cloud-sandbox provider | Implemented but not live-validated |
The same fixture-only provider contract supports explicit
--provider e2b selection, bounded uploads, deny-all egress
configuration plus a required outbound-denial probe, evidence normalization, and
cleanup safeguards. No E2B credential or successful E2B run was available for this
checked-in revision, so the recorded evidence below remains Docker-backed.
|
| Scoped authorization and durable local approvals | Functional locally | Browser and CLI share a pure policy evaluator. Synthetic persona scopes, separation of duties, append-only decisions, expiry, and hash-bound resume are tested; identity is not production OAuth/OIDC. |
| Deterministic governed context packs | Functional locally | Versioned public/synthetic records are selected by stage, tag, scope, freshness, priority, and budget rules. Per-stage manifests expose exclusions and bind artifacts, approvals, and local sandbox evidence to a SHA-256 digest. No semantic or vector retrieval is claimed. |
| Optional scoped local model gateway | gateway implemented; live provider path not validated | A typed offline adapter and an explicit LiteLLM local profile enforce model allow lists, scoped virtual credentials, fallback order, token/cost/time budgets, model-call traces, and cleanup. No live provider credential or successful live run was available for this revision. The static site never calls the gateway. |
| Jira, GitHub, LLM, database, enterprise MCP, review, and deployment operations | Simulated | No provider credentials, writes, or live data calls exist. |
| Anonymous or browser-invoked hosted sandbox execution | Simulated | Workbench logs remain deterministic fixtures and no hosted or browser-accessible sandbox is claimed. |
| Copy, download, export, confirmed reset, and state deep links | Functional | Clipboard copy and browser-generated files work locally; exported content is classified synthetic/public and contains no credentials or private data. |
Controlled execution evidence
One narrow local vertical slice proves allow-listed file modification, network-disabled container validation, measured command receipts, cleanup, and hash-verifiable evidence without exposing execution to public visitors.
Validated checked-in evidence
This is a build-time rendering of a recorded local Docker run. The visitor's browser does not execute code, start a container, accept a patch, or call a local service.
4d27eee7eafb · working tree cleansrc/report.js · only approved targetf3e9eab160e884695df71e4f218730e5932fbab39cbd8768b6ca4e15b1876c7713428edb9370abf9ff467da0f39b2028ecce63f0461fd47aed89b1f82aaed2731aa77a6ee7abf5c8ce8f5e6703f5125512dc204696355623306c71bbd354156e0286e362fba4a9ff3219ba846415150dsha256:adae508da516a8ea332e1b2ed1177ae1b35960a7225d3db1c233b09a9408a8fa| Command | Result | Duration basis |
|---|---|---|
| pre-test | Failed · exit 1 | 606 ms · measured locally |
| build | Passed · exit 0 | 401 ms · measured locally |
| test | Passed · exit 0 | 554 ms · measured locally |
diff --git a/src/report.js b/src/report.js
index dbf3576..3270318 100644
--- a/src/report.js
+++ b/src/report.js
@@ -1,3 +1,5 @@
export function formatVariance(actual, budget) {
- return `Variance: ${actual - budget}`;
+ const difference = actual - budget;
+ const direction = difference >= 0 ? "over" : "under";
+ return `Variance: ${Math.abs(difference)} ${direction}`;
}
Release controls
Release claims are generated only after an immutable commit passes local and hosted gates. The one-file evidence child names that audited parent, while the deployed build publishes and verifies its exact Vercel Git commit.
fc29578430771c1c06b8e5c6| Runtime image | Pinned source | Scanned digest | Provenance |
|---|---|---|---|
| Sandbox | ai-delivery-workbench-sandbox:node-22.23.1 |
sha256:adae508da516a8ea332e1b2ed1177ae1b35960a7225d3db1c233b09a9408a8fa |
digest pinned build |
| LiteLLM gateway | ai-delivery-workbench-litellm:v1.94.0-dev.3-database-mcp1.28.1 |
sha256:da0670d387ea5101d2b38ff47ad77472d0b6edb93666af9881a816be0701ebf6 |
verified upstream signature |
| PostgreSQL database | postgres:17.10-alpine3.24@sha256:742f40ea20b9ff2ff31db5458d127452988a2164df9e17441e191f3b72252193 |
sha256:93aa428db0aeeb71d24dcad1491bef6e1396a4255697e4bfc4c725bfeb981b74 |
digest pinned |
Recorded 2026-07-21T00:23:18.477Z against audited source-tree digest 8ac3b772401f3c63f9ca4e131d69c8263345e1fc6d597bcf3c87f0b2dde136a4. The evidence commit is a direct child whose only changed path is this generated summary. Vercel verified deployed commit 1c1c06b8e5c6973604b025b63aafed606b2bd522; the complete binding is published at deployment-binding.json. Detailed SARIF, SBOM, inventory, and scanner metadata stay in local or CI artifacts rather than the public bundle.
| Control | Recorded status | Tool | Findings |
|---|---|---|---|
| Tracked files and Git history secret scan | Executed | Gitleaks 8.30.1 | 0 |
| TypeScript, JavaScript, and container configuration static analysis | Executed | ESLint + repository container policy 10.7.0 / 1.1.0 | 0 |
| Locked npm dependency vulnerability scan | Executed | npm audit npm/11.12.0 | 0 |
| Runtime container vulnerability scans | Executed | Trivy 0.70.0 | 0 |
| Runtime image signature and provenance policy | Executed | Cosign + digest/build policy 3.1.2 / 1.0.0 | 0 |
| CycloneDX software bills of materials | Executed | CycloneDX npm + Trivy 6.0.0 / 0.70.0 | 0 |
| Dependency license inventory and policy | Executed | CycloneDX inventory + repository policy 1.0.0 | 0 |
| GitHub CodeQL JavaScript/TypeScript analysis | Executed | GitHub CodeQL v4 @ eec0bff2f6c1 Hosted run | 0 |
Selected implementation details
These excerpts are extracted from marker-bounded source at build time, so the case study fails its build if an excerpt drifts away from its implementation.
A discriminated action union limits the reducer to known state changes.
export type Action =
| { type: "ROUTE"; route: Route; key?: string }
| { type: "SELECT_ISSUE"; key: string; route?: Route }
| { type: "SET_STAGE"; key: string; idx: number; status: StageStatus }
| { type: "PATCH_ISSUE"; key: string; patch: IssuePatch }
| { type: "STALE_FROM"; key: string; fromIdx: number }
| { type: "INVALIDATE_DOWNSTREAM_AFTER_REDO"; key: string; stageId: StageId }
| { type: "TOAST_ADD"; toast: Toast }
| { type: "TOAST_LEAVE"; id: string }
| { type: "TOAST_REMOVE"; id: string }
| { type: "DRAWER"; drawer: LogDrawer | null }
| { type: "MODAL"; modal: WorkbenchModal | null }
| { type: "SELECT_ARTIFACT"; key: string; name: string }
| { type: "ARTIFACT_REVIEW"; artifactId: string; status: string }
| { type: "SET_SUBVIEW"; subview: string }
| { type: "PR"; key: string; patch: PullRequestOverride }
| { type: "VAL"; key: string; patch: ValidationOverride }
| { type: "BUSY"; id: string; on: boolean }
| { type: "FILTER"; patch: Partial<QueueFilters> }
| { type: "TOGGLE_GOV"; id: string }
| { type: "APPLY_SCENARIO"; scenarioId: DemoScenarioId }
| { type: "SET_PERSONA"; personaId: PersonaId }
| { type: "SET_APPROVAL_STORE"; approvalStore: ApprovalStore }
| { type: "AUTHORIZATION_DECISION"; decision: AuthorizationDecision | null }
| { type: "RESET" };
/*
Open store.tsx
A re-run marks completed dependent stages stale and records the issue-level condition.
case "STALE_FROM": {
const issue = state.issues[action.key];
if (!issue) return state;
const stages = issue.s.map((status, index) =>
index >= action.fromIdx && status !== "none" ? "stale" : status,
);
return {
...state,
issues: {
...state.issues,
[action.key]: {
...issue,
s: stages,
flags: { ...issue.flags, staleDownstream: true },
},
},
};
}
/*
Open store.tsx
Artifact metadata binds type, source stage, risk, timestamp, and review status.
export type ArtifactType = "JSON" | "Markdown";
export type ArtifactMeta = { type: ArtifactType; stage: StageId; risk: Risk };
export type Artifact = {
id: string;
name: string;
type: ArtifactType;
stage: string;
stageId: StageId;
timestamp: string;
reviewStatus: string;
risk: Risk;
lang: "json" | "md";
body: string;
contextPackDigest?: string;
};
/*
Open types.ts
Every proposed file is categorized and compared with the expected change surface.
export type PullRequestFile = {
path: string;
category: string;
add: number;
del: number;
status: "expected" | "unexpected";
};
/*
Open types.ts
Approval for validation refuses to advance until a person records changed-file review.
const approveForValidation = async () => {
const gate = evaluateValidationApproval({ issue, pullRequest: base, override: ov });
const blocker = gate.blockers[0];
if (blocker) {
const message = {
VERIFY_NOT_PASSED: [
"Verification must pass",
"A failed, stale, or incomplete Verify stage blocks validation approval.",
],
REQUIRED_CHECKS_NOT_PASSED: [
"Required checks must pass",
"Resolve every required automated check before requesting validation approval.",
],
DIFF_NOT_REVIEWED: [
"Review the diff first",
"Mark the changed-file diff reviewed before approving for validation.",
],
REVIEW_CHECKLIST_INCOMPLETE: [
"Complete the demo checklist",
"Every browser-local reviewer checklist item must be checked before validation approval.",
],
BOUND_APPROVAL_REQUIRED: ["Bound approval required", "A bound approval is required."],
VALIDATION_PERSONA_REQUIRED: ["Validator required", "A distinct validator is required."],
VALIDATION_SCENARIOS_INCOMPLETE: [
"Validation incomplete",
"Validation scenarios remain open.",
],
VALIDATION_EVIDENCE_INCOMPLETE: [
"Evidence incomplete",
"Validation evidence remains open.",
],
VALIDATION_ACCEPTANCE_INCOMPLETE: [
"Acceptance incomplete",
"Acceptance criteria remain open.",
],
VALIDATION_SECURITY_INCOMPLETE: [
"Security review incomplete",
"Security review remains open.",
],
VALIDATION_ACCESSIBILITY_INCOMPLETE: [
"Accessibility review incomplete",
"Accessibility review remains open.",
],
} as const;
actions.toast("warn", message[blocker.code][0], message[blocker.code][1]);
return;
}
const now = new Date().toISOString();
const validation = validationFor(issue);
const argumentsValue = {
issueKey: issue.key,
testedCommit: validation.commitSha,
decision: "release-ready",
} as const;
const context = await authorizeRegistryAction({
snapshot: registrySnapshot,
persona: personaById(state.personaId),
stageId: "review",
toolId: "tool.workflow.release-readiness",
targetPaths: [],
approvedChangeTargets: [],
networkAccess: false,
evidenceFinalized: true,
effectiveSubject: issue.key,
runId: `run.browser.${issue.key.toLocaleLowerCase()}`,
sessionId: "session.browser.demo",
now,
expiresAt: new Date(Date.parse(now) + 15 * 60 * 1000).toISOString(),
decisionId: `decision.browser.release.${issue.key.toLocaleLowerCase()}`,
});
actions.recordAuthorizationDecision(context.decision);
if (context.decision.mode !== "REQUIRE_APPROVAL") {
actions.toast(
context.decision.allowed ? "success" : "error",
context.decision.allowed ? "Transition allowed" : "Transition blocked",
context.decision.explanation,
);
return;
}
const changeTargetDigest = await sha256Hex(base.files.map((file) => file.path).sort());
const contextPackDigest = (await buildContextPack(issue.key, "review")).packDigest;
const request = await actions.queueApproval({
context,
argumentsValue,
targetPaths: [],
changeTargetDigest,
contextPackDigest,
requestedAt: now,
});
actions.toast(
"info",
"Validation approval requested",
`${request.requestId} is waiting for a distinct synthetic validator; no transition ran.`,
);
actions.navigate("approvals", issue.key);
};
/*
Open GitHubScreen.tsx
Productionization plan
These are design requirements, not capabilities attributed to the public demo.
Map people and service principals to scoped personas, approvals, and separation-of-duties policy.
Issue short-lived, per-agent credentials; never place reusable provider secrets in prompts or browsers.
Run tools in disposable workspaces with resource limits, network policy, and explicit egress.
Give actions stable identities, distinguish safe retries, and reconcile ambiguous provider outcomes.
Persist approvals and immutable evidence envelopes separately from mutable workflow projections.
Export OpenTelemetry-compatible traces and enforce measured-versus-estimated token, cost, time, and iteration budgets.
Support revocation, kill switches, recovery playbooks, evidence retention, and post-incident review.
Technical writing
Failure modes, policy boundaries, invalidation, and the tradeoff between autonomy and evidence.
Professional context — separate from this prototype
In professional work, I built a related governed AI-assisted delivery platform that supported approximately 50 production stories through human-reviewed pull requests. This public prototype is a separate implementation and contains none of that system’s code or data.
Clean-room disclosure
Independent portfolio prototype. All code, copy, fixtures, workflows, and visuals in this project were created from scratch using synthetic data. No employer or client code, prompts, schemas, screenshots, repositories, internal documentation, or confidential information were used. External Jira, GitHub, AI, database, and enterprise MCP-style operations are simulated; the interactive UI, local workflow state machine, and bounded toy-repository MCP fixture are functional. The public browser never connects to the local MCP process.
Public design influences and independent-development decisions are documented in the repository. Functional measurements, synthetic fixtures, and the professional-context statement remain three separate evidence categories.
Inspect the workflow