Description
- Vendor vs. Reality: GPT-6 Astra launches with "AGI era" branding, a perfect ExploitBench score, and 98.6% ARC-AGI-3 — but Simon Willison's teardown reveals custom harnesses and a 2.5x price premium drove those numbers. Artificial Analysis pegs Astra at an Intelligence Index of 61, dead even with its predecessor.
- Harnesses Get Built for You: ByteDance's HarnessDev and HarnessEvolve show open models constructing their own runtimes from empty sandboxes, while DeepSeek's Engram formalizes n-gram speculative decoding at 1.5-1.8x throughput. The orchestration layer is becoming a model capability, not a developer artifact.
- Benchmarks Are Broken: A systematic review of fifteen major agentic benchmarks finds none score safety, none track cost, and thirteen rely solely on binary task completion. New tools like VAKRA and IT-Bench shift focus to diagnosing why agents fail, while OpenEnv consolidates as the community-governed socket for agentic RL.
- Reliability Gets Quantified: Trajectory length emerges as the single most consequential design variable, and 307 hand-confirmed cases show adding skills made agents worse. Open models like Holo3.1 deliver 140ms local computer use on 12GB GPUs — crossing the production line from demo to deployment.
- Access Economics Bite: OpenAI pulls models from Cursor by November 12, GPT-6 won't make the model picker, and NVIDIA's $12.9B Hugging Face buyout casts a shadow over ZeroGPU grants. Capability is no longer the bottleneck — methodology, reliability, and access are.
Tags
AMDAmazonAnthropicAppleArena.aiArtificial Analysis+55 more
294 time saved2115 sources44 min read