ai-safe2-framework

AI SAFE² CLI Roadmap to 1.0

This roadmap organizes development by complete user capability rather than by date. Engineering pull requests may merge frequently. A public CLI release is created only when its capability gate is satisfied and the combined workflow has passed acceptance testing.

The objective for CLI 1.0 is a stable, provider-neutral evidence and decision layer for assessing agent systems. After 1.0, ordinary growth should primarily come from adapters, integrations, new assessment content, and compatible schema extensions rather than unfinished core workflows.

This is a planning document. Items below a released version are not delivered capabilities until their implementation, validation, and release evidence exist.

Technology-profile implementation proposal

The Technology Contribution Profile, assurance rubric, and candidate backlog provide v1.0 assessment methods. A machine schema, CLI command, adapter, and dashboard view remain unimplemented, separately scoped work. Their capability gate requires claim-level provenance, strict unknown/N/A states, validated control IDs, separation from organizational scoring and current E0-E5 weights, explicit translation/migration rules, negative tests, and end-to-end review. This proposal is not assigned to a released CLI version and does not delay the active release path.

Release policy

Current foundation

The CLI foundation through 0.6.0 includes project and MCP assessment, skill gating, AISM decision support, local environment discovery, NEXUS and NVIDIA SkillSpector evidence adapters, provider-neutral harness intake, task receipts, usage evidence, unified manifests, Challenge Lab fixture workflows, complete agent-system identity, evidence-bounded failure localization, deployment assessment scope, trusted-baseline change attribution, and technical release-readiness cards.

These capabilities retain their documented limits. They do not by themselves prove execution, root cause, billing, control effectiveness, organizational maturity, or framework conformance.

CLI 0.6: assess the correct change and deployment boundary (delivered)

User outcome: Determine what system is being assessed, what a change affected, and which findings are inherited, introduced, resolved, excluded, or unknown.

Delivered engineering sequence:

  1. Assessment scope contract
    • Define deployable subjects and component boundaries.
    • Classify product code, tests, adversarial fixtures, examples, research, generated artifacts, dependencies, and third-party components.
    • Record included, excluded, partial, missing, and not-applicable coverage.
  2. Change attribution
    • Compare a trusted baseline with the proposed revision.
    • Separate inherited findings from introduced, changed, and resolved findings.
    • Bind affected controls and components to the system identity manifest.
  3. Release-readiness card
    • Combine scoped findings, before/after evidence, failure localization, hosted checks, residual risks, ownership, and next actions.
    • Produce canonical agent JSON and a readable human card.

Release gate:

CLI 0.7: continuous task evidence and operational truth (delivered)

User outcome: Understand what agent work consumed, what it produced, what was verified, and where evidence disappeared across harness boundaries.

Delivered engineering sequence:

The delivered continuous surface is an explicit bounded one-shot command for operator-controlled local schedulers and CI. It does not install a daemon or claim native integrations that do not yet exist. Provider-specific adapters remain post-1.0 integration work unless a core abstraction requires them.

Release gate:

CLI 0.8: AISM implementation decisions and remediation

Status: Implemented in CLI 0.8.0; validation and release workflow complete before merge.

User outcome: Translate system and operational evidence into an actionable, human-owned improvement decision.

Planned capabilities:

Release gate:

CLI 0.9: Challenge Lab execution and independent evidence exchange

Status: Implemented for release-candidate validation

User outcome: Run reproducible agent-governance experiments and compare independent implementations without forcing them into identical internal semantics.

Planned capabilities:

Release gate:

CLI 0.9.1–0.9.9: stable-core release candidate

Status: Implemented in the ordered release-candidate stack; exact combined revision validation and independent acceptance remain release gates.

This sequence closes the highest-value pre-1.0 usability and evidence gaps:

  1. secure-default project initialization and explicit configuration precedence;
  2. one bounded safe2 assess project workflow;
  3. provider-neutral adapter descriptors, evidence, and conformance reports;
  4. privacy-preserving Codex JSONL reference translation;
  5. offline OpenTelemetry OTLP/JSON trace import and non-content metadata export;
  6. explicit continuous local change evidence with immutable per-run reports;
  7. claim/receipt contradiction auditing without speculative honesty scoring;
  8. offline installed-runtime and packaged-contract self-checks; and
  9. a replayable first-user acceptance bundle that disclaims independence.

These additions make the core adaptable across harnesses while retaining facts, provider reports, gaps, and decisions as separate layers. The Codex and OpenTelemetry adapters consume explicit files; they are not ambient session access, native harness hooks, or enforcement integrations.

CLI 1.0: stable core and integration platform

Status: Release qualification implemented on the 1.0 release branch. Final status depends on the exact release revision’s hosted checks and accountable release-owner decision; this roadmap is not itself a release declaration.

User outcome: Install one supported CLI and obtain a complete, reviewable picture of an agent system’s safety, security, governance, compliance evidence, operational performance, and unresolved decisions.

Core completion requirements:

The 1.0 release gate requires:

  1. All required hosted checks pass on the exact release revision.
  2. Clean-package and documented-workflow acceptance passes across the supported runtime matrix.
  3. Security, usability, adaptability, modularity, compatibility, and evidence honesty reviews have no unresolved release-blocking findings.
  4. Agent acceptance covers multiple harness perspectives; human acceptance covers accountable security, engineering, governance, and first-time users.
  5. The AI SAFE² self-assessment and at least one attributed independent review are recorded with their scope and limitations.
  6. Every planned core capability is delivered, explicitly deferred as non-core, or removed from the 1.0 claim by the release owner.
  7. Release notes provide a verified before/after view, quick start, limitations, rollback path, and next integration priorities.

After 1.0

The ranked, capability-gated continuation is maintained in CLI post-1.0 priorities. The first priority is a guided agent onboarding workflow; the next layer adds admission hooks and maintained native adapters without weakening the provider-neutral core.

Post-1.0 work should normally extend the stable core through:

An integration that exposes a missing core abstraction may justify a 1.x core change. It should not silently break the 1.0 contracts or redefine prior evidence.

Decision discipline

The roadmap is complete only when the user outcome is complete. A merged pull request, passing test count, generated card, third-party score, or published announcement is evidence of one step. None substitutes for the capability gate of the target release.