Why lower latency can leave timely task delivery unchanged
Every task in a four-task cohort falls from 12 to 11 seconds, yet timely completions stay at zero. With a 10-second deadline, that is entirely possible. Another change is easier to misread: leave raw latencies untouched, mark two slow tasks as failures, and successful-task median latency falls from 9 to 5.5 seconds.
Engineers evaluating multi-agent serving need different explanations for those two improvements. One fails to cross a delivery threshold; the other changes the statistical population. Internal module controls separate the mechanisms here. They help readers check deadlines, counts, percentile cohorts and measurement windows; they do not identify a serving optimization already shown to be faster.
Four faster tasks can still yield no more timely deliveries
The same-date research executed the pinned MAServeBench task-metric module with four synthetic tasks, a 20-second measurement duration and a 10-second deadline. Every task is marked successful; paired controls change only raw wall-clock latency. The seconds are stipulated array values, not completion times measured on a server.
| Internal synthetic control | Latency per task | Attempted | Successful | Timely | Task goodput |
|---|---|---|---|---|---|
| Baseline input | 12 seconds | 4 | 4 | 0 | 0 tasks/s |
| Faster but still late | 11 seconds | 4 | 4 | 0 | 0 tasks/s |
| At the deadline boundary | 10 seconds | 4 | 4 | 4 | 0.2 tasks/s |
Source: same-date internal baseline, faster-but-late and deadline-boundary module controls; all inputs are synthetic.
The code counts a successful task as timely when latency is no greater than the deadline, then divides timely completions by the caller-supplied duration. The first two rows have different latencies but identical threshold counts. At 10 seconds, the inclusive predicate counts all four tasks, giving 4/20 = 0.2 tasks/s.
This does not make a reduction from 12 to 11 seconds worthless. It means the reduction does not automatically fulfill a 10-second delivery commitment. Continuous latency statistics alone cannot answer a deadline-delivery question. Conversely, a threshold count does not describe how severely late tasks miss their deadline; both observables have a role.
A lower median can reflect the loss of slow successes
The second pair retains raw latencies of 5, 6, 12 and 20 seconds. All four tasks succeed in the first input. Only the 12- and 20-second tasks are marked failed in the second. No task executes faster.
| Internal synthetic control | Successful | Successful-task median | Attempted | Timely | Attainment |
|---|---|---|---|---|---|
| All four successful | 4 | 9 seconds | 4 | 2 | 0.5 |
| Two slow tasks marked failed | 2 | 5.5 seconds | 4 | 2 | 0.5 |
Source: same-date internal before/after success-selection controls. This module calculates percentiles from valid successful-task latencies and attainment over all recorded attempts.
The improved median with unchanged attainment now comes from selection, rather than a deadline plateau. These mechanisms cannot be combined into a speedup. Successful-task percentiles remain useful when labeled as such and accompanied by attempted, successful and timely counts.
A further internal control supplies a 5-second success, a 6-second failure, a task marked successful with missing latency, and a 12-second success. The module returns four attempts, three successes, one timely task and attainment 0.25. Failures and missing latency remain in the attempted denominator. That denominator nevertheless covers recorded attempts only; the module cannot discover attempts that callers never recorded.
The clock and run validity remain separate conditions
Tasks can include external waits. A control supplies both 12 seconds of raw wall-clock latency and 1 second of net latency. The pinned module prefers the raw time and still reports zero timely tasks. Removing a wait from a local timing statistic does not shorten arrival-to-completion time experienced by a recipient.
Positive metrics also cannot establish run validity. An internal control marks a run invalid while supplying one successful 5-second task; the module still emits 0.05 tasks/s and attainment 1. Such numbers may be diagnostic without qualifying the run for a valid performance comparison. A separate capacity-helper control supplies three synthetic load cells: load 1 is valid and meets the target; load 2 meets it numerically but is invalid; load 3 is valid but misses it. At target 0.9, the helper returns maximum tested qualifying load 1, already filtering the invalid cell. This is not an executed load sweep or evidence of an implementation defect.
The caller also owns the observation window. The v2 paper's complete-task counting protocol distinguishes arrival-window goodput from a drain-inclusive timely-task rate. The first uses the fixed arrival window; the second and token throughput use the larger of that window and cohort completion duration. A module receiving one duration parameter cannot reconstruct the required arrival cohort and drain window by itself.
Paper performance findings occupy a different evidence layer
The module mechanisms relate to the question raised by Section 6.6 and the high-load table in the v2 paper, but do not reproduce its hardware experiment. At the highest load, the authors report that raising the DAG execution's KV budget from 16 to 40 GiB gives paired-block median ratios of 1.0774 for token throughput and 0.8451 for root-task P95 latency. Median deadline goodput remains near zero, and one paired block loses one timely root.
Those author-reported findings cannot be inserted into the four-task table or compounded with the request-limit sensitivity experiment. They have distinct loads, hardware/model conditions, paired units and deadlines. The paper also retains low-load, loose- or strict-deadline ties and binding-dependent results. Our interpretation supports possible divergence among metrics, rather than a universal ranking of release policies or cache configurations.
Evidence and sources
This article uses completed same-date research. Nine internal controls executed and passed; their synthetic inputs, expectations and actual outputs accompany the candidate. The metric module is pinned to public commit 22483ab5df5eff562061953d564aac47a9423c2f. The sole first-run assertion discrepancy was floating-point representation: expected 5.99 versus actual 5.989999999999999. Analysis preserved that run and changed only numerical comparison tolerance before passing. Production did not rerun or add experiments.
The task-metric code is the public primary implementation of the rules. Our execution is internal experimental evidence without an independent reproducer. Hardware findings remain author-reported from the v2 paper. The pinned reproducibility documentation states that the cleaned public core excludes original machine-specific result archives; mock fixtures cannot substitute for paper hardware results.
Publication is not validation; citation is not endorsement; peer review is not proof. Self-reported evidence is not independent evidence, and successful module execution does not establish general performance validity.
What a causal optimization claim would still require
We lack the original frozen corpus, private result archives and complete model/hardware serving stack. The probe measures no request throughput, real arrival process, KV cache, replay executor or hardware profile. Its three capacity cells are synthetic summaries. Current public commits and original paper experiment versions must remain separately identified.
A serving evaluation can first retain a common set of readings: complete-task raw wall-clock latency, attempted/successful/timely counts, a pre-locked deadline, a named observation window and run validity. Repeated native/replay comparisons on an obtainable frozen corpus and specified hardware would then need to examine whether bindings, arrival variation and model changes alter the result. That evidence is needed to answer whether an optimization delivers more tasks on time or improves a narrower statistic alone.