agent brief/2026-09-28

The Harness Is the Product

Across sources, practitioners converge on one claim: agent reliability is engineered in the runtime, not the model — and the benchmarks keep proving it.

time to read56m
time saved113 min
sources1.3k
The Harness Is the Product
λsynopses
  • Reliability Moves Outward LangGraph tops framework comparisons for observability and HITL, while tool-calling, tracing and guardrail guidance all place enforcement in the runtime.
  • Benchmarks Crack A vLLM stress test reportedly drops Llama-3.1-70B tool-selection accuracy from 95% to 20% as catalogs grow; sub-1B routers are the proposed fix.
  • Quants Hide Damage One user's identical 4-bit AWQ runs on DeepSWE diverged by ~7 points and solved different task sets.
#tags
subscribe
system operational
end :: 1,291 signals processed█
keep reading
→recent briefs
2026-11-16

Agent Threat Meets Agent Scale

- **Capability Push** DeepSeek-V4 claims a million-token context for agents; the usable retrieval curve, not the window, is the real question. - **Security Edge** The FBI's ARTEX advisory and Microsoft's Storm-0940 attribution to a ConnectWise compromise remain vendor and threat-intel claims. - **Infra Matures** OpenEnv's Linux Foundation handoff signals agent tooling consolidating — while the attack surface sharpens.

2026-11-13

Agents Get Safer, Faster, Harder to Grade

- **Idempotency Goes Default** Builders converge on retry-safe actions and approval gates before shipping side effects. - **Speed Over Frontier** Gemini 4 Hydrogen edges Argon 51–50 on Artificial Analysis' index while finishing tasks 2.5x faster. - **Benchmarks Reset** Hugging Face claims GAIA's top spot as Meta's Gaia2 reframes evaluation around acting, best score 42% pass@1.

2026-11-12

Approval Must Bind the Call

- **Gates Go Granular** Reddit builders converge on one lesson: approval must bind to the exact tool call, not the session or a timer. - **Breach Postmortem** Hugging Face's timeline shows an agent with legitimate tool access chaining permissions into an incident; Microsoft, OpenAI, Akka and Scalekit describe invocation-level gates. - **Memory Math** MosaicLeaks reports leakage worsening from 34% to 51.7% when agents train purely for task performance — prompting alone may not hold.