Agent Brief
by .agent community

Tag

Peking University

2 issues found

Nov 11, 2026

Benchmarks Meet Their Limits

Description

  • Orchestration Gap Einsia's WorkflowArena holds the worker model fixed and varies only the planner, arguing coordination is separate from raw model power — authors' own claim, not an independent re-run.
  • Scorecard Asterisks H Company's Holo4 claims 85.2% on OSWorld for its research-only 27B, while the Apache-2.0 35B-A3B scores 80.8% — all vendor-reported.
  • Failure Taxonomies MAST's 14 failure modes and VAKRA's "structural" multi-step tool-chain failures point practitioners at harness and design, with reports of harness choice halving token usage.

Tags

AWSAgentOpsAlibabaAmazonAnthropicArize+64 more
286 time saved529 sources29 min read

Nov 9, 2026

Agents Learn to Decide

Description

  • Open Weights Rise: Thinking Machines' Inkling 2 Small reportedly scored 81.7% on ARC-AGI-2 at $0.27/task, topping ARC Prize's open-weight evals.
  • Cheap Decisions: Decision models like Drex Max claim 183ms answers versus Opus 5.5's 3.5s, reshaping routing economics for builders.
  • Prove It Twice: Holo4 adds a third action lane, but IBM, Microsoft and authorization-drift work push repeatability and boundaries.

Tags

ARC PrizeAirbnbAmazonAndreessen HorowitzAnthropicBaidu+45 more
795 time saved5060 sources24 min read