Microsoft on Tuesday, June 3, 2026, unveiled ASSERT — Adaptive Spec-driven Scoring for Evaluation and Regression Testing — an open source framework designed to help developers test whether AI systems behave as intended within specific applications and products.
The framework works by taking plain-language descriptions of an AI model’s expected behaviors and policies, converting them into structured sets of acceptable and unacceptable behaviors, generating test cases and problem scenarios, running those tests against the target system, and scoring the results. ASSERT also records the paths an AI system takes — including intermediate actions and tool calls — so developers can trace where failures occur.
Developers can provide additional system context, tools, and constraints to further customize evaluations. As an example, Microsoft describes a scenario where a developer specifies that a document research AI agent should not send emails outside the company, should restrict confidential information to C-level executives, and should provide concise summaries with prior context in mind. ASSERT would then generate ongoing tests to check whether the system follows those rules.
Sarah Bird, chief product officer of Responsible AI at Microsoft, said the tool addresses a gap that broader evaluations cannot fill. “If you really want to have a trustworthy system, you should evaluate many more dimensions that are application-specific,” Bird said. She added that ASSERT can be used during development, after deployment, and for continuous monitoring.
Microsoft says the framework responds to a specific need that has emerged as AI models become more capable: ensuring that AI systems behave according to the particular context, policies, and tools of a given product or service — something general-purpose evaluations are not built to address.
The release comes as the broader AI industry has been expanding its focus on repeatable testing and regression checks, with organizations including Stanford’s HELM, MLCommons’ AILuminate, and evaluation group METR developing benchmarks to measure model behavior under varying conditions.
Source: TechCrunch