Skip to content
OBSERVATION NOTES

Open-source Engineering

Runtime, workflow, recovery, tools, skills, observability and governance mechanisms.

132observation notes
Observation note list132 shown, sorted newest first
01
Daily ObservationExperiment ReportAwaiting review

Fewer calls do not establish honest stopping: an isolated scoring experiment

Ninety-six internal controlled cases expose how the released scorer blends semantic judgment and call-count rewards. The failed rank-reversal prediction is retained, without inferring real model behavior.

→
02
Daily ObservationExperiment ReportAwaiting review

The AI says the human approved it. Can it act for them?

An Orca run treated an Agent message as human permission. We tested six authorization-source and request-binding controls at CodeFlowMu's preparation interface.

→
03
Daily ObservationExperiment ReportAwaiting review

The ticket was escalated. Why did the AI answer about shipping?

A tool says the ticket was escalated just as the user asks about an order. In a bounded AG2 before/after comparison, one extra model call replaces the handoff result with a shipping answer. We compare that turn boundary with CodeFlowMu's delivery evidence.

→
04
Daily ObservationProject ResearchAwaiting review

BoundaryXP: Make an AI Team’s Task Boundaries Visible

Agents plan steps, use tools, and delegate work. Their plans can also expand beyond the task they were given. BoundaryXP uses 7+1 to organize requirements and mathematical methods to check budgets, permissions, ordering, and relationships across tasks.

→
05
Daily ObservationExperiment ReportAwaiting review

The order went through. The AI worker saw only a timeout. Should it retry?

The business system committed an order while the AI worker saw a timeout. We killed a process at three boundaries and used a blind retry without an idempotency key as a control.

→
06
Daily ObservationExperiment ReportAwaiting review

When AI checks AI, who checks the evidence it saw?

A report has the right number of items, the current evidence version, and valid authorization, yet says B-23 where the task asked for B-22. Seven controlled cases show what rules and two models can catch, and what they cannot establish.

→
07
Daily ObservationExperiment ReportAwaiting review

After the Causal Label: What an Attribution Module Probe Establishes

Nonempty placeholder chains and nonexistent endpoints can activate a causal label in a pinned attribution module. Twelve synthetic calls expose its local acceptance predicate and functioning negative controls, without measuring real-run misattribution rates.

→
08
Daily ObservationTechnical AnalysisAwaiting review

Before Reading a Collusion Score, Ask Who Survived the Label Rule

Strict labels combine target votes with private stance. Mixed signals are excluded and missing stance is nonpositive; score interpretation needs evidence completeness and the retained population.

→
09
Daily ObservationExperiment ReportAwaiting review

The task never moved. Why does its record say it entered review?

A controlled submission failed while the task stayed in the active directory, yet its metadata already said review. We compared retry and lost-receipt paths to ask how an agent team proves a transition completed.

→
10
Daily ObservationExperiment ReportAwaiting review

When grading writes into the action record

Six internal substituted-environment probes of pinned original methods show that reward evaluation can append reference actions to the action record, including through a terminal step. Mutation can affect downstream provenance attribution but does not itself establish an incorrect reward; historical prevalence, repair effects and current-version behavior remain unverified.

→