One Lab
Book a demo

Open Agent Observatory · Read-only program

Public facts, bounded runtime evidence.

Assure observes the exact ten-project registry without changing pins or executing upstream code. Runtime evidence remains attached to one commit, suite, environment, and receipt—never to an entire third-party project.

10/10 runtime-evidenced · 4 pin updates available · 0 licence-file changes · observed Sep 13, 2026, 4:56:14 a.m. UTC

Download the canonical ten-project observation· Assure production run

01 / Registry and drift

Ten projects stay in the denominator.

A daily low-cost metadata job compares each default-branch head and licence-file blob with the registered commit. An upstream move creates UPSTREAM_UPDATE_AVAILABLE; it never changes a pin and never means verified. Controlled runtime evidence is scoped to the exact registered pin and the stated framework, UI-wiring, or research-harness boundary.

agent

mini-swe-agent

SWE-agent/mini-swe-agentRUNTIME_REPRODUCED_AT_PIN
Latest observed upstream
04d809ceab9df28f9adaed044884180159172930
NO_PIN_DRIFT
Latest release / tag
RELEASE · v2.4.6
Licence
MIT · drift NO
Adapter surfaces
subprocess · container · python-hook
Last check
2026-09-13T04:56:14.232Z
Runtime proof
FRAMEWORK_RUNTIME_REPRODUCED_AT_PIN
Controlled runs
6
Last successful run
mini-swe-agent:success:v1
2026-09-04T00:50:43.293726Z
Failure cases
contradiction · loop · permission-refusal · timeout · tool-failure
Sentinel coverage
v0 FROZEN_DEVELOPMENT · v1 PROSPECTIVE_ONLY
Claim boundary
framework runtime only; public/synthetic deterministic tasks; untested capabilities remain UNKNOWN
Scope: public/synthetic deterministic task
Assure observation receiptsha256:ad98c…Latest controlled runtime receiptsha256:52916…6 controlled runtime receiptscontradiction · sha256:64d4a…loop · sha256:29a2c…permission-refusal · sha256:117eb…success · sha256:52916…timeout · sha256:6ff37…tool-failure · sha256:1c2cb…
agent

Browser Use

browser-use/browser-usePIN_DRIFT
Latest observed upstream
50f205533fe10ba35b553d2a3689c77b87bd5d0a
UPSTREAM_UPDATE_AVAILABLE
Latest release / tag
RELEASE · 0.13.10
Licence
MIT · drift NO
Adapter surfaces
python-hook · browser-observer
Last check
2026-09-13T04:56:14.232Z
Runtime proof
FRAMEWORK_RUNTIME_REPRODUCED_AT_PIN
Controlled runs
6
Last successful run
browser-use:form-success:v1
2026-09-04T00:53:01.154893Z
Failure cases
loop · navigation-error · permission-refusal · timeout
Sentinel coverage
v0 FROZEN_DEVELOPMENT · v1 PROSPECTIVE_ONLY
Claim boundary
framework runtime only; public/synthetic deterministic tasks; untested capabilities remain UNKNOWN
Scope: public/synthetic deterministic task
Assure observation receiptsha256:94f33…Latest controlled runtime receiptsha256:f20b0…6 controlled runtime receiptsform-success · sha256:f20b0…loop · sha256:9d690…navigation-error · sha256:c20f9…navigation-success · sha256:7ab43…permission-refusal · sha256:ffffe…timeout · sha256:6fe59…
agent

CrewAI

crewAIInc/crewAIPIN_DRIFT
Latest observed upstream
894898f84c4ac0a89f24bf7bee6c381eb0e67f51
UPSTREAM_UPDATE_AVAILABLE
Latest release / tag
RELEASE · 1.15.21
Licence
MIT · drift NO
Adapter surfaces
python-hook · callback · otlp
Last check
2026-09-13T04:56:14.232Z
Runtime proof
FRAMEWORK_RUNTIME_REPRODUCED_AT_PIN
Controlled runs
6
Last successful run
crewai:sequential-success:v1
2026-09-04T00:50:54.433539Z
Failure cases
contradiction · coordination-failure · timeout · tool-failure
Sentinel coverage
v0 FROZEN_DEVELOPMENT · v1 PROSPECTIVE_ONLY
Claim boundary
framework runtime only; public/synthetic deterministic tasks; untested capabilities remain UNKNOWN
Scope: public/synthetic deterministic task
Assure observation receiptsha256:c6526…Latest controlled runtime receiptsha256:451a2…6 controlled runtime receiptscontradiction · sha256:fded5…coordination-failure · sha256:35381…delegation-success · sha256:d5e9a…sequential-success · sha256:451a2…timeout · sha256:11929…tool-failure · sha256:14b92…
agent

smolagents

huggingface/smolagentsRUNTIME_REPRODUCED_AT_PIN
Latest observed upstream
30bb1161095dbae2271e6bc3cc4c219cc3897a57
NO_PIN_DRIFT
Latest release / tag
RELEASE · v1.26.0
Licence
Apache-2.0 · drift NO
Adapter surfaces
python-hook · callback
Last check
2026-09-13T04:56:14.232Z
Runtime proof
FRAMEWORK_RUNTIME_REPRODUCED_AT_PIN
Controlled runs
6
Last successful run
smolagents:code-success:v1
2026-09-04T00:50:20.609706Z
Failure cases
loop · permission-refusal · timeout · tool-failure
Sentinel coverage
v0 FROZEN_HELD_OUT · v1 PROSPECTIVE_ONLY
Claim boundary
framework runtime only; public/synthetic deterministic tasks; untested capabilities remain UNKNOWN
Scope: public/synthetic deterministic task
Assure observation receiptsha256:560a3…Latest controlled runtime receiptsha256:fa1f5…6 controlled runtime receiptscode-success · sha256:fa1f5…loop · sha256:b98ae…permission-refusal · sha256:71c4f…timeout · sha256:9e180…tool-failure · sha256:bb419…tool-success · sha256:6e4e0…
agent

Microsoft Agent Framework

microsoft/agent-frameworkPIN_DRIFT
Latest observed upstream
3c670707766a8455da6491a9049cc9d575e019f0
UPSTREAM_UPDATE_AVAILABLE
Latest release / tag
RELEASE · dotnet-1.21.0
Licence
MIT · drift NO
Adapter surfaces
otlp · mcp · callback
Last check
2026-09-13T04:56:14.232Z
Runtime proof
FRAMEWORK_RUNTIME_REPRODUCED_AT_PIN
Controlled runs
1
Last successful run
controlled-runtime-v2:agent-framework:2026-09-13T04:37:25.607676Z
2026-09-13T04:37:25.607676Z
Failure cases
client-failure-observed
Sentinel coverage
v0 NOT_IN_V0 · v1 PROSPECTIVE_ONLY
Claim boundary
native controlled wiring only; no model performance, project-wide verification, calibration, or current-upstream claim
Scope: Agent.run with a framework-native chat-client interface and deterministic local responses
Assure observation receiptsha256:a08ae…Latest controlled runtime receiptsha256:f9cc3…
agent

OpenHands

OpenHands/OpenHandsPIN_DRIFT
Latest observed upstream
28464621d879e3e9b3ceeae9d70a71d96da6212d
UPSTREAM_UPDATE_AVAILABLE
Latest release / tag
RELEASE · v1.18.0
Licence
MIT · drift NO
Adapter surfaces
container · subprocess · browser-observer
Last check
2026-09-13T04:56:14.232Z
Runtime proof
AGENT_UI_RUNTIME_WIRING_REPRODUCED_AT_PIN
Controlled runs
1
Last successful run
controlled-runtime-v2:openhands:2026-09-13T04:37:53.698Z
2026-09-13T04:37:53.698Z
Failure cases
failed-command-normalisation
Sentinel coverage
v0 NOT_IN_V0 · v1 PROSPECTIVE_ONLY
Claim boundary
native controlled wiring only; no model performance, project-wide verification, calibration, or current-upstream claim
Scope: registered Agent Canvas pin runtime-service delegation and failure-result normalization tests; no agent backend or model execution
Assure observation receiptsha256:f7f15…Latest controlled runtime receiptsha256:bbb09…
research harness

BrowserGym

ServiceNow/BrowserGymRUNTIME_REPRODUCED_AT_PIN
Latest observed upstream
9e779f087de9a65668b6974d11f9ce9816026e96
NO_PIN_DRIFT
Latest release / tag
RELEASE · v0.14.3
Licence
Apache-2.0 · drift NO
Adapter surfaces
python-hook · browser-observer
Last check
2026-09-13T04:56:14.232Z
Runtime proof
RESEARCH_HARNESS_RUNTIME_REPRODUCED_AT_PIN
Controlled runs
1
Last successful run
controlled-runtime-v2:browsergym:2026-09-13T04:37:45.098703Z
2026-09-13T04:37:45.098703Z
Failure cases
invalid-action-observed
Sentinel coverage
v0 NOT_IN_V0 · v1 PROSPECTIVE_ONLY
Claim boundary
native controlled wiring only; no model performance, project-wide verification, calibration, or current-upstream claim
Scope: BrowserGym open-ended environment against a local synthetic page with system Chrome
Assure observation receiptsha256:267ec…Latest controlled runtime receiptsha256:7e612…
research harness

AgentLab

ServiceNow/AgentLabRUNTIME_REPRODUCED_AT_PIN
Latest observed upstream
cbc35a9bc0facaf731bc858c5825edbe757c719f
NO_PIN_DRIFT
Latest release / tag
RELEASE · v0.4.2
Licence
Apache-2.0 · drift NO
Adapter surfaces
python-hook · browser-observer
Last check
2026-09-13T04:56:14.232Z
Runtime proof
RESEARCH_HARNESS_RUNTIME_REPRODUCED_AT_PIN
Controlled runs
1
Last successful run
controlled-runtime-v2:agentlab:2026-09-13T04:37:48.907183Z
2026-09-13T04:37:48.907183Z
Failure cases
parse-failure-observed
Sentinel coverage
v0 NOT_IN_V0 · v1 PROSPECTIVE_ONLY
Claim boundary
native controlled wiring only; no model performance, project-wide verification, calibration, or current-upstream claim
Scope: MostBasicAgent action selection with a deterministic local chat-model adapter and synthetic observation
Assure observation receiptsha256:03467…Latest controlled runtime receiptsha256:790d3…
research harness

AgentDojo

ethz-spylab/agentdojoRUNTIME_REPRODUCED_AT_PIN
Latest observed upstream
089ed468cf3ed0322acc66b0211f26d9d90dbf60
NO_PIN_DRIFT
Latest release / tag
RELEASE · v0.1.35
Licence
MIT · drift NO
Adapter surfaces
python-hook · callback
Last check
2026-09-13T04:56:14.232Z
Runtime proof
RESEARCH_HARNESS_RUNTIME_REPRODUCED_AT_PIN
Controlled runs
1
Last successful run
controlled-runtime-v2:agentdojo:2026-09-13T04:37:34.935197Z
2026-09-13T04:37:34.935197Z
Failure cases
unknown-tool-failure-observed
Sentinel coverage
v0 NOT_IN_V0 · v1 PROSPECTIVE_ONLY
Claim boundary
native controlled wiring only; no model performance, project-wide verification, calibration, or current-upstream claim
Scope: FunctionsRuntime registration and execution using synthetic state and arguments
Assure observation receiptsha256:4d0d1…Latest controlled runtime receiptsha256:e5d99…
research harness

DoomArena

ServiceNow/DoomArenaRUNTIME_REPRODUCED_AT_PIN
Latest observed upstream
b80902f107b4d28194580352a59b3029f4a018b4
NO_PIN_DRIFT
Latest release / tag
TAG · v0.3.0
Licence
Apache-2.0 · drift NO
Adapter surfaces
python-hook · browser-observer
Last check
2026-09-13T04:56:14.232Z
Runtime proof
RESEARCH_HARNESS_RUNTIME_REPRODUCED_AT_PIN
Controlled runs
1
Last successful run
controlled-runtime-v2:doomarena:2026-09-13T04:37:35.646346Z
2026-09-13T04:37:35.646346Z
Failure cases
NO CONTROLLED FAILURE CASE IN THIS RECEIPT
Sentinel coverage
v0 NOT_IN_V0 · v1 PROSPECTIVE_ONLY
Claim boundary
native controlled wiring only; no model performance, project-wide verification, calibration, or current-upstream claim
Scope: native fixed-injection and bounded sequence attack primitives with synthetic strings only
Assure observation receiptsha256:e9960…Latest controlled runtime receiptsha256:4e512…

02 / Observation receipt

The displayed state has a source.

Observed at
2026-09-13T04:56:14.232Z
Snapshot receipt
sha256:0eff9990924dbe0a43bf6c8095583a470b24e6b9e0d84583643035a7d925dd1e
Assure commit
78406a24fa89511cc3c42d1b79d7ce9ee305de37
Artifact ID
10311368272
Artifact file SHA-256
bc5680e1d870fcba8f3feeeced46aebe81cb58f6fa5e85ad998eefcfd545e7a3
Controlled runtime receipt root
sha256:d60c13ed9460c3354196f0bd688ae1d277ac5922602ae028184eb3d9505d96cb
Controlled runtime production artifact
10312185847
Authority
read-only · no pin updates · no upstream execution · Sentinel OBSERVE/WARN only

Runtime manifest· Production provenance

03 / Corpus integrity

Historical runtime evidence stays frozen.

The v0 corpus still binds 24 receipts for the original four exact-pin framework runs. Its ephemeral test signature is explicitly not an assurance anchor.

Receipt root
sha256:d3f1b6675e6963e952f48e7420bd1dd5fc72db5adc7aadb286585312704806ee
Sentinel evaluation
sha256:b7420c8add90979b4b14d2a0ac5408740d335cb55ec4a22a73e45b2c16c4a8b6
Signature class
ONE_LAB_EPHEMERAL_TEST_SIGNER_NOT_AN_ASSURANCE_ANCHOR

Corpus manifest ·Sentinel evaluation JSON

04 / Sentinel v0

Uncalibrated and permanently frozen.

mini-swe-agent, Browser Use, and CrewAI formed development; smolagents was entirely held out. The result is not retuned as upstreams move.

Held-out runs / failures
6 / 4
TP / FP / TN / FN
1 / 2 / 0 / 3
Precision / recall
33.3% / 25.0%
Confidence
UNCALIBRATED

Claim: early-warning detection under these controlled public/synthetic experiments. It does not establish predictive performance.

05 / Verification boundary

A receipt binds one result.

Runtime reproduction means framework wiring executed for the named project, exact commit, task suite, and environment. It is not model-performance validation, certification, endorsement, or project-wide verification.

  • ASSURE_VERIFIED_RUNTIME
  • REPRODUCED_BY_ONE_LAB
  • SELF_REPORTED_BY_PROJECT
  • EXTERNALLY_OBSERVED_PUBLIC
  • UNKNOWN

06 / Safety

Observe and warn only.

Metadata checks are read-only. Controlled experiments use public or synthetic tasks; raw prompts and outputs stay off, credential-shaped values fail closed, and no customer data enters the corpus. Sentinel cannot intervene, block, or mutate a customer system.