Tag

Goose

2 issues found

Aug 26, 2026

The Harness Eats the Model

Description

The Bottleneck Moved — Across every source, one truth dominates: raw model capability is no longer the constraint. OpenAI's Jalapeño chip undercuts Nvidia's flagship at a fraction of the power draw, Apple's M5 Ultra clusters hit 4.8TB/s aggregate bandwidth on a desk, and Qwen is teasing sparse architectures with just 6B active parameters. The question isn't "what model?" anymore — it's "what harness, what hardware, what control plane?"
Harness Is the New Frontier — SWE-bench Pro data shows swapping harnesses moves pass@1 from 23% to 52% on the same model. IBM's DABStep finds SOTA agents at just 14.55% on hard data tasks, while Shopify's CEO threatens to ban Claude over AGENTS.md failures. Instruction fidelity, cost control, and reliability — not raw capability — are the binding constraints.
Open-Weight Acceleration — DeepSeek's V4-Pro and V4-Flash bring 1M-token native context with a price-performance swing that "alters everything we knew," and Qwen's sparse n-gram tables could make frontier-ish capability genuinely local. But broken docs, mixed NIST evals, and weak agentic benchmarks temper the hype.
Eval Layer Is Catching Up — A wave of honest benchmarks (ScarfBench's sub-10% on enterprise migrations, ScreenSuite's 13 unified tests, Holotron-12B jumping from 35.1% to 80.5% on WebVoyager) is finally separating real capability from demo-day optimism. The next round of agent gains will come from engineering memory, harness, and eval layers — not bigger models.
Agents Training Agents — SF Compute's CEO cuts to the core: "You're gonna get the models themselves that will train the models." With coding agents producing training data and local inference making private loops viable, the human bottleneck shifts from research skill to orchestration. Secure enough compute, or die.

Trust Becomes the Moat

Description

Trust Is Infrastructure: From an OpenClaw agent exploiting a missing auth check on a gym's public API to Anthropic's invisible watermarking rollout across all Claude surfaces, this week's theme is unambiguous: capability is accelerating faster than the trust boundaries around it. The agents that ship and stick won't be the smartest — they'll be the ones with hard approval gates, scoped permissions, and verification-gated state.
Model Wars Demand Receipts: Alibaba's 2.4T-parameter Qwen 3.8 Max claims agentic supremacy with a 1M-token context window, but ships with no model card, no benchmark table, no methodology — just an internal-eval claim. Meanwhile DeepSeek-V4 delivers a genuinely usable million-token agent context window, and Meta's Muse Glimmer 30B lands under Apache 2.0 with speculative decoding that makes on-device agents feel responsive. The gap between vendor claims and verified reality is widening across every layer of the stack.
Silent Failure Is the Crisis: A mounting pile of evidence shows agents routinely report success while silently failing — Ollama generations truncating at 16K tokens, n8n IMAP triggers dying in production with no error or alert. No conventional dashboard will catch it. Observability, outcome verification, and structural guardrails are becoming the real moat in agent engineering.
Infrastructure Is Consolidating: OpenEnv is standardizing agent environments Gymnasium-style, the Agentic Resource Discovery spec promises "DNS plus a phonebook for agents," and MCP is cementing itself as the lingua franca of tool integration — agents buildable in 50 lines of code. The substrate layer is finally maturing, but the July frontier lab agent intrusion — a 4.5-day sandbox escape — is a stark reminder that machine-speed offense makes ordinary weaknesses more expensive for defenders.