| Framework Home | Cross-Pillar Governance | AISM | NEXUS | Dashboard |
Record challenge results in the Technology Contribution Profile with claim-level assurance. Keep C0-C5 claim maturity, E0-E5 assurance v1.0, technology contribution 0-5, and organizational AISM maturity separate; there is no automatic conversion. The validation guide and nine-example backlog frame new composed-capability, process-mimicking, side-effect, and policy-integrity tests. They do not modify frozen Challenge 001 or claim those experiments have run.
AI governance controls should not receive credit because they sound appropriate, appear in a policy, or map to a standard. They should receive credit when a reproducible implementation changes a security-relevant outcome under test.
The Challenge Lab converts incidents, evaluation findings, and credible agentic threats into open experiments. Each challenge publishes the problem, hypotheses, control claims, baselines, graders, evidence requirements, invalidation criteria, and replication path before confirmatory results are promoted.
The framework is the subject of the test, not the source of its own proof.
Every challenge should be able to answer:
| ID | Challenge | Primary question | Status |
|---|---|---|---|
| 001 | Anthropic Multi-Agent Turf War | Can externally enforced authority stop destructive multi-agent conflict without blocking legitimate collaboration? | Live study: design and pre-registration; offline CLI fixture available |
Start with the executable Challenge CLI guide. The offline runner verifies fixture mechanics only. Its synthetic third-party example demonstrates translation, not independent evidence of control effectiveness.
| Level | Status | Evidence |
|---|---|---|
| C0 | Described | Control language exists |
| C1 | Implemented | Machine-readable configuration and code exist |
| C2 | Unit verified | Positive and negative unit tests pass |
| C3 | Scenario validated | Control changes an adversarial scenario outcome |
| C4 | Bypass tested | Control survives replay, corruption, direct bypass, and fail-mode testing |
| C5 | Independently replicated | An unaffiliated operator reproduces the material result |
Only C3 and above support the phrase validated in tested conditions. No maturity level establishes universal prevention.
Challenge maturity is separate from framework/profile conformance. A well-designed experiment may invalidate a conformant implementation claim, and a conformant implementation may fail to outperform a simpler baseline.
AI SAFE² v3.1 distinguishes:
A challenge result must identify which plane was tested. Evidence from one plane does not automatically validate another.
Every challenge includes:
Material normative or grader changes after preregistration require a new preregistration version before confirmatory evidence is pooled.
An invalidated claim is a useful result. It should create a control revision, permanent regression fixture, or versioned record of why the claim changed.
High-value contributions include:
The most important contribution is evidence that changes what the framework can honestly claim.
| Framework Home | Challenge 001 | Cross-Pillar Governance | AISM | NEXUS | Contributing | Security |
AI SAFE² principle: If governance is not enforced at runtime, it is not governance.
AI SAFE² v3.1 · Cyber Strategy Institute