Tag
Dries Buytaert
2 issues found
Nov 11, 2026
Benchmarks Meet Their Limits
Description
- Orchestration Gap Einsia's WorkflowArena holds the worker model fixed and varies only the planner, arguing coordination is separate from raw model power — authors' own claim, not an independent re-run.
- Scorecard Asterisks H Company's Holo4 claims 85.2% on OSWorld for its research-only 27B, while the Apache-2.0 35B-A3B scores 80.8% — all vendor-reported.
- Failure Taxonomies MAST's 14 failure modes and VAKRA's "structural" multi-step tool-chain failures point practitioners at harness and design, with reports of harness choice halving token usage.
Tags
AWSAgentOpsAlibabaAmazonAnthropicArize+64 more
286 time saved529 sources29 min read
Oct 1, 2026
Agent Platforms, Manager-Worker Splits
Description
- Platform Land Grab — Anthropic, OpenAI (Dots, GPT-6.1 Sol, Spaces, Agents API) and Grok (Bot, Muse) all pushed agent platforms in one cycle, with @MLStreetTalk alleging ecosystem lock-in intent.
- Orchestration Pays — Anthropic's own test reportedly shows Fable 5 orchestrating Sonnet 5 workers at 96% of all-Fable performance for 46% of the cost, echoing Meta's manager-worker compute finding.
- Cost Reality — OpenAI's $200 Pro drops from 20× to 10× Plus reportedly on October 30, 2026, plus a $500 "Pro 500" tier; unreplicated MoE offload hits 50-100 tok/s locally while quadratic attention makes 2M context expensive.
Tags
AI EdgeLabsAMDAWS LabsAircallAlibabaAmazon+141 more
257 time saved1784 sources54 min read