Tag

Agent Reliability

2 issues found

Nov 11, 2026

Description

Orchestration Gap Einsia's WorkflowArena holds the worker model fixed and varies only the planner, arguing coordination is separate from raw model power — authors' own claim, not an independent re-run.
Scorecard Asterisks H Company's Holo4 claims 85.2% on OSWorld for its research-only 27B, while the Apache-2.0 35B-A3B scores 80.8% — all vendor-reported.
Failure Taxonomies MAST's 14 failure modes and VAKRA's "structural" multi-step tool-chain failures point practitioners at harness and design, with reports of harness choice halving token usage.

Description

Frontier Performance Meets Localism Zhipu AI's 744B GLM-5.2 is challenging GPT-5.5 performance, emphasizing the shift toward capable open-weights as US policy shifts tighten access to cloud-based frontier models.
Code-as-Action Over Brittle JSON The industry is pivoting from fragile JSON-based orchestration toward a Code-as-Action philosophy with frameworks like smolagents, aiming to solve the high failure rates seen in complex enterprise SRE scenarios.
Context Expansion and Determinism While subquadratic scaling pushes context windows to a staggering 12 million tokens, practitioners are moving away from vibe-based development toward rigorous adversarial review loops and automated validation gates.
Standardizing the Developer Stack Vercel’s new Agent Stack and the Cursor Doctrine signify a maturation of the ecosystem, focusing on durable workflows, long-running sandboxes, and protocol-level code editing.