Tag

Microsoft Research

3 issues found

Nov 11, 2026

Benchmarks Meet Their Limits

Description

  • Orchestration Gap Einsia's WorkflowArena holds the worker model fixed and varies only the planner, arguing coordination is separate from raw model power — authors' own claim, not an independent re-run.
  • Scorecard Asterisks H Company's Holo4 claims 85.2% on OSWorld for its research-only 27B, while the Apache-2.0 35B-A3B scores 80.8% — all vendor-reported.
  • Failure Taxonomies MAST's 14 failure modes and VAKRA's "structural" multi-step tool-chain failures point practitioners at harness and design, with reports of harness choice halving token usage.

Tags

AWSAgentOpsAlibabaAmazonAnthropicArize+64 more
286 time saved529 sources29 min read

Oct 12, 2026

Local Models Meet Agent Reality Check

Description

  • Local Qwen Surge Qwen3.8-27B reportedly hit 3M downloads in three days, though VRAM alone can't fix inference-engine bottlenecks.
  • Agent Reality Check Gartner forecasts 40%+ of agentic projects canceled by end-2027 on cost and weak risk controls.
  • Code-First Agents Hugging Face re-architected its stack so models write code; vendor-reported 80.4% on OSWorld-Verified lacks neutral testing.
  • Benchmark Hardening IBM and UC Berkeley work shifts from leaderboards to failure taxonomies.

Tags

ARMOAembitAgentic PatternsAgoraAmazonAnthropic+107 more
434 time saved610 sources28 min read

Jun 17, 2026

Persistent Memory and Open-Weight Surge

Description

  • The End of Ephemerality Vercel’s new Agent Stack and projects like Recall are shifting agents from stateless functions to persistent, stateful systems capable of 24-hour workflows.
  • Open-Weights Reach Parity GLM-5.2 and DeepSeek-V4 are shattering records, offering frontier-level reasoning and 1M-token context windows that challenge proprietary API dominance.
  • Minimalist Orchestration Wins Hugging Face’s smolagents is proving that "Code-as-Action" outperforms heavy DAG frameworks by slashing JSON parsing overhead and tool-calling loops.
  • Regulatory and Safety Volatility Anthropic’s export control withdrawals and "invisible" safety interventions emphasize the need for sovereign, local-first AI infrastructure.

Tags

AlibabaAnthropicBerkeleyCoinbaseDeepSeekGoogle+33 more
324 time saved2156 sources18 min read