Skip to case study

Independent engineering case study

AI Delivery Workbench

A governed, human-in-the-loop control plane for AI-assisted software delivery.

An inspectable workflow that constrains agent actions and context, requires human decisions, and binds evidence to the tested change.

Functional here
Navigation, local workflow state, approvals, exports, and evidence validation.
Simulated here
Jira, GitHub, model, database, deployment, and every external write.

Seven-minute guided walkthrough

Follow one authorization and evidence chain.

Start with the thesis, drill into one synthetic issue, make a distinct human decision, and finish at the recorded sandbox evidence and production boundaries.

  1. Issuestage state
  2. Run manifestagent · tool · context · budget
  3. Human gatescoped reviewer
  4. Local sandboxallow-listed execution
  5. Evidence + tracetested tree · receipts · spans
  6. PR readinesshuman transition
The browser replays checked-in evidence; it never executes Docker or accepts visitor code. Every tour step has a resumable deep link.

The problem

Generation alone is not a delivery system.

A coding agent can produce a plausible patch while the surrounding process remains unsafe, stale, or impossible to audit. The workbench treats those control failures as first-class product problems.

01

Nondeterministic output

Equivalent prompts can produce materially different plans, patches, and explanations.

02

Stale or excessive context

Old decisions and unbounded repository data can quietly distort the next stage.

03

Unsafe direct writes

Broad credentials collapse suggestion, execution, and authorization into one action.

04

Weak provenance

Without inputs, versions, and digests, an artifact cannot explain how it came to be.

05

Detached validation

Evidence that is not bound to the tested commit can outlive the code it actually covered.

06

Self-approval

An AI system must not be the sole authority that accepts its own risky work.

Design principles

Make control visible in the workflow.

  1. Prefer deterministic artifacts over transient chat.Each stage emits a durable, reviewable handoff.
  2. Invalidate downstream work after upstream changes.Stale state is explicit; previous success is not silently reused.
  3. Bound context and tool permissions.Inputs and actions are selected for the stage, not granted globally.
  4. Keep branch and pull-request controls authoritative.Agent output enters the same engineering control path as other changes.
  5. Put people in control of transitions.Risky actions wait for an explicit human decision with scoped authority.
  6. Attach validation evidence to the tested commit.Review can verify which code, checks, and assumptions support a decision.

System workflow

Eight stages, one chain of custody.

A stage advances only when its required input, review state, and policy gates agree. A revised upstream artifact marks dependent work stale instead of pretending the old result remains valid.

  1. 01SeedEmits a pinned issue and revision.Blocks on missing source identity.
  2. 02IntakeEmits normalized scope and constraints.Blocks on ambiguity or excluded context.
  3. 03SpecEmits acceptance criteria and risks.Blocks on incomplete human review.
  4. 04PlanEmits ordered work and dependencies.Blocks on unresolved design choices.
  5. 05Change TargetsEmits the expected file surface.Blocks on disallowed or unclear targets.
  6. 06ImplementEmits a branch patch and provenance.Blocks on tool, policy, or budget limits.
  7. 07VerifyEmits commit-bound test evidence.Blocks on failed or incomplete checks.
  8. 08PR ReviewEmits a human-owned decision.Blocks on scope drift or missing evidence.

Architecture

Separate steering from execution.

Four responsibility planes keep workflow policy, runtime activity, selected context, and validation evidence independently inspectable.

Control PlaneState · policy · approvals · budgets
Execution PlaneStage runtimes · bounded tools · workspaces
Context PlaneManifests · provenance · freshness · exclusions
Validation PlaneChecks · review decisions · evidence envelopes
The control plane authorizes execution, selects bounded context, and requires validation evidence before a human-controlled transition.
Architecture diagram text alternative

The Control Plane owns workflow state, agent and tool policy, approvals, and budgets. It authorizes the Execution Plane to run a stage with bounded tools; selects a manifest from the Context Plane; and requires the Validation Plane to attach check results and human decisions to the change. Execution does not approve its own output.

Illustrative adapter set

Jira, GitHub, Angular, .NET, and Oracle are one synthetic reference stack behind vendor-neutral issue, repository, UI, service, and database boundaries. They are not a required environment and no external adapter is contacted by the demo.

Functional versus simulated

Claims follow observable evidence.

The matrix distinguishes repository behavior from synthetic representations and future work.

Functional, simulated, and planned capabilities in the public prototype.
Capability Classification Evidence boundary
Case-study navigation and demo routing Functional Static routes and validated browser-local deep links for screens, synthetic issues, artifacts, Settings subviews, scenarios, and the guided walkthrough.
Workflow transitions, retry, redo, and stale behavior Functional Typed reducer state, confirmed deterministic reset, and synthetic fixtures.
Artifact, pull-request, and validation views Functional UI Viewing and local review state work; all displayed records and metrics are synthetic.
Versioned agent/tool registry and capability cards Functional JSON Schema validation, SHA-256 content hashes, approved-version gating, manifest invalidation, search/detail UI, and generated public JSON are exercised in this repository.
Repository-owned local MCP toy-repository slice Functional locally A repository command discovers and invokes bounded tools over stdio in a disposable synthetic repository, then emits sanitized static evidence. The browser never connects to the process.
Explicit local Docker validation slice Functional locally A developer command applies one deterministic allow-listed patch to a temporary synthetic repository, runs fixed build/tests with container networking disabled, and emits validated recorded evidence. The public site cannot invoke it.
Optional E2B cloud-sandbox provider Implemented but not live-validated The same fixture-only provider contract supports explicit --provider e2b selection, bounded uploads, deny-all egress configuration plus a required outbound-denial probe, evidence normalization, and cleanup safeguards. No E2B credential or successful E2B run was available for this checked-in revision, so the recorded evidence below remains Docker-backed.
Scoped authorization and durable local approvals Functional locally Browser and CLI share a pure policy evaluator. Synthetic persona scopes, separation of duties, append-only decisions, expiry, and hash-bound resume are tested; identity is not production OAuth/OIDC.
Deterministic governed context packs Functional locally Versioned public/synthetic records are selected by stage, tag, scope, freshness, priority, and budget rules. Per-stage manifests expose exclusions and bind artifacts, approvals, and local sandbox evidence to a SHA-256 digest. No semantic or vector retrieval is claimed.
Optional scoped local model gateway gateway implemented; live provider path not validated A typed offline adapter and an explicit LiteLLM local profile enforce model allow lists, scoped virtual credentials, fallback order, token/cost/time budgets, model-call traces, and cleanup. No live provider credential or successful live run was available for this revision. The static site never calls the gateway.
Jira, GitHub, LLM, database, enterprise MCP, review, and deployment operations Simulated No provider credentials, writes, or live data calls exist.
Anonymous or browser-invoked hosted sandbox execution Simulated Workbench logs remain deterministic fixtures and no hosted or browser-accessible sandbox is claimed.
Copy, download, export, confirmed reset, and state deep links Functional Clipboard copy and browser-generated files work locally; exported content is classified synthetic/public and contains no credentials or private data.

Controlled execution evidence

Recorded real sandbox run

One narrow local vertical slice proves allow-listed file modification, network-disabled container validation, measured command receipts, cleanup, and hash-verifiable evidence without exposing execution to public visitors.

Validated checked-in evidence

Failing before, passing after one allow-listed patch.

This is a build-time rendering of a recorded local Docker run. The visitor's browser does not execute code, start a container, accept a patch, or call a local service.

SUCCEEDED
Source commit
4d27eee7eafb · working tree clean
Changed path
src/report.js · only approved target
Execution boundary
0.5 CPU · 256 MiB · 64 processes · network none
Evidence digest
f3e9eab160e884695df71e4f218730e5932fbab39cbd8768b6ca4e15b1876c77
Context-pack digest
13428edb9370abf9ff467da0f39b2028ecce63f0461fd47aed89b1f82aaed273
Trace ID
1aa77a6ee7abf5c8ce8f5e6703f51255
Trace artifact digest
12dc204696355623306c71bbd354156e0286e362fba4a9ff3219ba846415150d
Budget outcome
WARNING | 3 tool calls | 1 repair attempt
Model accounting
0 calls | $0 | exact zero no model
Sandbox runtime
sha256:adae508da516a8ea332e1b2ed1177ae1b35960a7225d3db1c233b09a9408a8fa
Measured command results from the recorded local Docker run.
CommandResultDuration basis
pre-test Failed · exit 1 606 ms · measured locally
build Passed · exit 0 401 ms · measured locally
test Passed · exit 0 554 ms · measured locally
Inspect the recorded unified diff
diff --git a/src/report.js b/src/report.js
index dbf3576..3270318 100644
--- a/src/report.js
+++ b/src/report.js
@@ -1,3 +1,5 @@
 export function formatVariance(actual, budget) {
-  return `Variance: ${actual - budget}`;
+  const difference = actual - budget;
+  const direction = difference >= 0 ? "over" : "under";
+  return `Variance: ${Math.abs(difference)} ${direction}`;
 }

Release controls

Security and supply-chain evidence

Release claims are generated only after an immutable commit passes local and hosted gates. The one-file evidence child names that audited parent, while the deployed build publishes and verifies its exact Vercel Git commit.

Release tagv1.0.8
Audited commitfc2957843077
Deployed commit1c1c06b8e5c6
Runtime image SBOMs3
Active suppressions15
Runtime imagePinned sourceScanned digestProvenance
Sandbox ai-delivery-workbench-sandbox:node-22.23.1 sha256:adae508da516a8ea332e1b2ed1177ae1b35960a7225d3db1c233b09a9408a8fa digest pinned build
LiteLLM gateway ai-delivery-workbench-litellm:v1.94.0-dev.3-database-mcp1.28.1 sha256:da0670d387ea5101d2b38ff47ad77472d0b6edb93666af9881a816be0701ebf6 verified upstream signature
PostgreSQL database postgres:17.10-alpine3.24@sha256:742f40ea20b9ff2ff31db5458d127452988a2164df9e17441e191f3b72252193 sha256:93aa428db0aeeb71d24dcad1491bef6e1396a4255697e4bfc4c725bfeb981b74 digest pinned

Recorded 2026-07-21T00:23:18.477Z against audited source-tree digest 8ac3b772401f3c63f9ca4e131d69c8263345e1fc6d597bcf3c87f0b2dde136a4. The evidence commit is a direct child whose only changed path is this generated summary. Vercel verified deployed commit 1c1c06b8e5c6973604b025b63aafed606b2bd522; the complete binding is published at deployment-binding.json. Detailed SARIF, SBOM, inventory, and scanner metadata stay in local or CI artifacts rather than the public bundle.

ControlRecorded statusToolFindings
Tracked files and Git history secret scan Executed Gitleaks 8.30.1 0
TypeScript, JavaScript, and container configuration static analysis Executed ESLint + repository container policy 10.7.0 / 1.1.0 0
Locked npm dependency vulnerability scan Executed npm audit npm/11.12.0 0
Runtime container vulnerability scans Executed Trivy 0.70.0 0
Runtime image signature and provenance policy Executed Cosign + digest/build policy 3.1.2 / 1.0.0 0
CycloneDX software bills of materials Executed CycloneDX npm + Trivy 6.0.0 / 0.70.0 0
Dependency license inventory and policy Executed CycloneDX inventory + repository policy 1.0.0 0
GitHub CodeQL JavaScript/TypeScript analysis Executed GitHub CodeQL v4 @ eec0bff2f6c1 Hosted run 0

Selected implementation details

The controls are represented in code.

These excerpts are extracted from marker-bounded source at build time, so the case study fails its build if an excerpt drifts away from its implementation.

Typed state transitions

A discriminated action union limits the reducer to known state changes.

export type Action =
  | { type: "ROUTE"; route: Route; key?: string }
  | { type: "SELECT_ISSUE"; key: string; route?: Route }
  | { type: "SET_STAGE"; key: string; idx: number; status: StageStatus }
  | { type: "PATCH_ISSUE"; key: string; patch: IssuePatch }
  | { type: "STALE_FROM"; key: string; fromIdx: number }
  | { type: "INVALIDATE_DOWNSTREAM_AFTER_REDO"; key: string; stageId: StageId }
  | { type: "TOAST_ADD"; toast: Toast }
  | { type: "TOAST_LEAVE"; id: string }
  | { type: "TOAST_REMOVE"; id: string }
  | { type: "DRAWER"; drawer: LogDrawer | null }
  | { type: "MODAL"; modal: WorkbenchModal | null }
  | { type: "SELECT_ARTIFACT"; key: string; name: string }
  | { type: "ARTIFACT_REVIEW"; artifactId: string; status: string }
  | { type: "SET_SUBVIEW"; subview: string }
  | { type: "PR"; key: string; patch: PullRequestOverride }
  | { type: "VAL"; key: string; patch: ValidationOverride }
  | { type: "BUSY"; id: string; on: boolean }
  | { type: "FILTER"; patch: Partial<QueueFilters> }
  | { type: "TOGGLE_GOV"; id: string }
  | { type: "APPLY_SCENARIO"; scenarioId: DemoScenarioId }
  | { type: "SET_PERSONA"; personaId: PersonaId }
  | { type: "SET_APPROVAL_STORE"; approvalStore: ApprovalStore }
  | { type: "AUTHORIZATION_DECISION"; decision: AuthorizationDecision | null }
  | { type: "RESET" };
/*
Open store.tsx

Stale downstream invalidation

A re-run marks completed dependent stages stale and records the issue-level condition.

case "STALE_FROM": {
      const issue = state.issues[action.key];
      if (!issue) return state;
      const stages = issue.s.map((status, index) =>
        index >= action.fromIdx && status !== "none" ? "stale" : status,
      );
      return {
        ...state,
        issues: {
          ...state.issues,
          [action.key]: {
            ...issue,
            s: stages,
            flags: { ...issue.flags, staleDownstream: true },
          },
        },
      };
    }
    /*
Open store.tsx

Artifact provenance shape

Artifact metadata binds type, source stage, risk, timestamp, and review status.

export type ArtifactType = "JSON" | "Markdown";
export type ArtifactMeta = { type: ArtifactType; stage: StageId; risk: Risk };
export type Artifact = {
  id: string;
  name: string;
  type: ArtifactType;
  stage: string;
  stageId: StageId;
  timestamp: string;
  reviewStatus: string;
  risk: Risk;
  lang: "json" | "md";
  body: string;
  contextPackDigest?: string;
};
/*
Open types.ts

Changed-file classification

Every proposed file is categorized and compared with the expected change surface.

export type PullRequestFile = {
  path: string;
  category: string;
  add: number;
  del: number;
  status: "expected" | "unexpected";
};
/*
Open types.ts

Human approval gate

Approval for validation refuses to advance until a person records changed-file review.

const approveForValidation = async () => {
    const gate = evaluateValidationApproval({ issue, pullRequest: base, override: ov });
    const blocker = gate.blockers[0];
    if (blocker) {
      const message = {
        VERIFY_NOT_PASSED: [
          "Verification must pass",
          "A failed, stale, or incomplete Verify stage blocks validation approval.",
        ],
        REQUIRED_CHECKS_NOT_PASSED: [
          "Required checks must pass",
          "Resolve every required automated check before requesting validation approval.",
        ],
        DIFF_NOT_REVIEWED: [
          "Review the diff first",
          "Mark the changed-file diff reviewed before approving for validation.",
        ],
        REVIEW_CHECKLIST_INCOMPLETE: [
          "Complete the demo checklist",
          "Every browser-local reviewer checklist item must be checked before validation approval.",
        ],
        BOUND_APPROVAL_REQUIRED: ["Bound approval required", "A bound approval is required."],
        VALIDATION_PERSONA_REQUIRED: ["Validator required", "A distinct validator is required."],
        VALIDATION_SCENARIOS_INCOMPLETE: [
          "Validation incomplete",
          "Validation scenarios remain open.",
        ],
        VALIDATION_EVIDENCE_INCOMPLETE: [
          "Evidence incomplete",
          "Validation evidence remains open.",
        ],
        VALIDATION_ACCEPTANCE_INCOMPLETE: [
          "Acceptance incomplete",
          "Acceptance criteria remain open.",
        ],
        VALIDATION_SECURITY_INCOMPLETE: [
          "Security review incomplete",
          "Security review remains open.",
        ],
        VALIDATION_ACCESSIBILITY_INCOMPLETE: [
          "Accessibility review incomplete",
          "Accessibility review remains open.",
        ],
      } as const;
      actions.toast("warn", message[blocker.code][0], message[blocker.code][1]);
      return;
    }
    const now = new Date().toISOString();
    const validation = validationFor(issue);
    const argumentsValue = {
      issueKey: issue.key,
      testedCommit: validation.commitSha,
      decision: "release-ready",
    } as const;
    const context = await authorizeRegistryAction({
      snapshot: registrySnapshot,
      persona: personaById(state.personaId),
      stageId: "review",
      toolId: "tool.workflow.release-readiness",
      targetPaths: [],
      approvedChangeTargets: [],
      networkAccess: false,
      evidenceFinalized: true,
      effectiveSubject: issue.key,
      runId: `run.browser.${issue.key.toLocaleLowerCase()}`,
      sessionId: "session.browser.demo",
      now,
      expiresAt: new Date(Date.parse(now) + 15 * 60 * 1000).toISOString(),
      decisionId: `decision.browser.release.${issue.key.toLocaleLowerCase()}`,
    });
    actions.recordAuthorizationDecision(context.decision);
    if (context.decision.mode !== "REQUIRE_APPROVAL") {
      actions.toast(
        context.decision.allowed ? "success" : "error",
        context.decision.allowed ? "Transition allowed" : "Transition blocked",
        context.decision.explanation,
      );
      return;
    }
    const changeTargetDigest = await sha256Hex(base.files.map((file) => file.path).sort());
    const contextPackDigest = (await buildContextPack(issue.key, "review")).packDigest;
    const request = await actions.queueApproval({
      context,
      argumentsValue,
      targetPaths: [],
      changeTargetDigest,
      contextPackDigest,
      requestedAt: now,
    });
    actions.toast(
      "info",
      "Validation approval requested",
      `${request.requestId} is waiting for a distinct synthetic validator; no transition ran.`,
    );
    actions.navigate("approvals", issue.key);
  };
  /*
Open GitHubScreen.tsx

Productionization plan

A credible prototype names its missing infrastructure.

These are design requirements, not capabilities attributed to the public demo.

Identity and authorization

Map people and service principals to scoped personas, approvals, and separation-of-duties policy.

Secrets and credentials

Issue short-lived, per-agent credentials; never place reusable provider secrets in prompts or browsers.

Isolated execution

Run tools in disposable workspaces with resource limits, network policy, and explicit egress.

Idempotency and retries

Give actions stable identities, distinguish safe retries, and reconcile ambiguous provider outcomes.

Durable state and evidence

Persist approvals and immutable evidence envelopes separately from mutable workflow projections.

Observability and cost controls

Export OpenTelemetry-compatible traces and enforce measured-versus-estimated token, cost, time, and iteration budgets.

Rollback and incident handling

Support revocation, kill switches, recovery playbooks, evidence retention, and post-incident review.

Technical writing

Governing AI-assisted delivery

Failure modes, policy boundaries, invalidation, and the tradeoff between autonomy and evidence.

Read the technical article

Professional context — separate from this prototype

Related experience, independently implemented here.

In professional work, I built a related governed AI-assisted delivery platform that supported approximately 50 production stories through human-reviewed pull requests. This public prototype is a separate implementation and contains none of that system’s code or data.

Clean-room disclosure

Independent by design.

Independent portfolio prototype. All code, copy, fixtures, workflows, and visuals in this project were created from scratch using synthetic data. No employer or client code, prompts, schemas, screenshots, repositories, internal documentation, or confidential information were used. External Jira, GitHub, AI, database, and enterprise MCP-style operations are simulated; the interactive UI, local workflow state machine, and bounded toy-repository MCP fixture are functional. The public browser never connects to the local MCP process.

Public design influences and independent-development decisions are documented in the repository. Functional measurements, synthetic fixtures, and the professional-context statement remain three separate evidence categories.

Inspect the workflow

Follow the evidence from intake to human review.