AI-news notes.

Anthropic's fastest model in the 3.5 family ships in November 2024 and immediately reframes what 'small' means in production AI. At 40.6% on SWE-bench Verified it outperforms the original Claude 3.5 Sonnet and GPT-4o on coding tasks — a result that would have been frontier-tier six months earlier, now running at the speed of the prior generation's cheapest model. For developers, the operational story is latency and throughput: Claude 3.5 Haiku matches Claude 3 Haiku's inference speed while surpassing Claude 3 Opus, Anthropic's previous flagship, across every intelligence benchmark. It's the model you route sub-agent tasks to when you need Claude-quality reasoning without paying Claude Sonnet prices. Pricing settled at $0.80/$4.00 per million tokens — a fraction of Sonnet costs — making it the default choice for high-volume pipelines, classification layers, and any task where you're paying per call at scale.

OpenAI opens the Realtime API to public beta in October 2024, giving developers a persistent WebSocket connection to GPT-4o that handles speech-to-speech in real time — no STT/LLM/TTS pipeline stitching required. The model speaks back in one of six preset voices, calls functions mid-conversation, and responds at latencies that feel like a phone call rather than a query. Audio pricing was steep at launch: $100/$200 per million input/output tokens (roughly $0.06 per minute in, $0.24 per minute out), which positioned it as infrastructure for voice products rather than a casual API feature. The architectural significance is what it replaces: until this API, building a real-time voice assistant meant routing audio through three separate services and accepting the compounding latency of each hop. Now it's one model, one connection, one latency budget. The Chat Completions API gained audio input/output support on October 17 as a companion feature. Every voice product roadmap in Q4 2024 was revised the week this shipped.

Apple's redesigned Mac Mini ships with M4 and M4 Pro chips at $600 and $1,400. The critical moment: a $600 machine can now run 70B parameter models at usable inference speeds via Ollama or LM Studio. Local AI stops being a hobbyist exercise and becomes a practical daily option. The on-device inference era begins in earnest.

Anthropic open-sources MCP — a universal connector standard for AI agents and external systems. Rapidly adopted by OpenAI, Google, and the major tooling providers. SDKs in Python, TypeScript, C#, Java. Before MCP, every tool integration was custom. After MCP, it's a standard protocol with a growing ecosystem of pre-built connectors.

Alibaba releases Qwen 2.5-Coder in 7B (Sep), 32B, and 72B (Nov) variants. The 32B beats GPT-4o and Claude 3.5 Sonnet on HumanEval. Open weights under Apache 2.0, runnable on a Mac Studio. The strongest evidence yet that specialized open models can outperform general-purpose proprietary ones on targeted tasks.

OpenAI ships the full o1 on December 5, 2024 — not a preview, a production model — and the gap over o1-preview is measurable: 34% reduction in major errors on hard problems, SWE-bench jumps from 41.3% to 48.9%, AIME math score from 42 to 79.2. The preview had already changed how people thought about reasoning; the full release changed how they deploy it. API access opens to developers on usage tier 5 at $15/$60 per million input/output tokens — expensive enough to stay out of hot paths, cheap enough to justify for the hard problems worth thinking about. The same day, OpenAI launches ChatGPT Pro at $200/month, the first premium tier above Plus, which includes o1 pro mode: a version of the model that uses more compute to think harder and is noticeably better on the hardest math and science problems. The safety posture improved in parallel: safety evaluation score went from 22 (o1-preview) to 84. The preview had made reasoning models a product category; the full release made o1 a reliable production choice.

Meta closes out 2024 on December 6 with Llama 3.3 70B Instruct — and the headline is efficiency, not scale: a 70B model that matches Llama 3.1 405B on the benchmarks that matter. IFEval instruction following is actually higher at 92.1% versus 88.6% for the 405B; MATH hits 77.0%, HumanEval 88.4%. The cost story is stark — Meta reports that 1 million output tokens cost $0.40 with Llama 3.3 70B versus $1.80 with the 405B, a 4.5× cost advantage for near-identical output quality. Eight languages, 128K context window, text-only instruction-tuned. The model mostly outperformed GPT-4o on benchmarks at release while running on hardware the 405B couldn't touch. The message is the same one that had defined 2024's open-weight story: every six months, the same benchmark score requires half the compute. Llama 3.3 70B is the clearest single proof point of that trend to date.

Google announces Gemini 2.0 Flash: native image and audio output, a Multimodal Live API for real-time interaction, tool use baked in. Beats Gemini 1.5 Pro at twice the speed. Marketed explicitly as built for 'the agentic era.' Free in AI Studio. The first Gemini model that felt like Google had caught up.

A 671B MoE model trained for $5.58M that matches Claude 3.5 Sonnet and o1 on most benchmarks and runs 3x faster than its predecessor. No CUDA dependency. A full technical paper published alongside it. The cost number — $5.58M — circled the internet for weeks and forced every lab to publicly rethink their training cost assumptions.

Meta ships Llama 3.1 with a 405B parameter flagship, 128K context, and support for eight additional languages. The 405B model is the first open-weight model that closes the gap with frontier proprietary models on coding tasks. Released under a license that allows commercial use for most companies. The moment open-source stopped being 'almost as good' and became genuinely competitive.

OpenAI releases GPT-4o mini on July 18, 2024 at $0.15/$0.60 per million input/output tokens — more than 60% cheaper than GPT-3.5 Turbo, and the most cost-efficient OpenAI model ever at that point. The benchmark profile explains why this matters beyond price: 82.0% on MMLU versus 77.9% for Gemini Flash and 73.8% for Claude Haiku; 87.0% on MGSM math reasoning; 87.2% on HumanEval coding. It beat both major competing small models on every benchmark at a lower price point. The model replaces GPT-3.5 Turbo as the default across ChatGPT's free tier and OpenAI's API recommendations for most production use cases — a deliberate commoditization of the tier below GPT-4o. The 128K context window and 16K output limit were generous for the price class. For developers, GPT-4o mini collapsed the usual tradeoff between running cheap models that perform poorly and expensive models that perform well: at this price point, there was no longer a practical reason to run a weaker open-source model through local inference unless your latency requirements or data constraints forced it.

Mistral releases Large 2 on July 24, 2024 — 123B parameters, 128K context, MIT license — their most capable open-weight model and a direct challenge to the proprietary mid-tier. The coding numbers are the headline: 92.0% on HumanEval, matching Claude 3.5 Sonnet and beating GPT-4o's coding score; 76.9% average across seven languages versus GPT-4o's 77.9%. MMLU lands at 84.0%. For multilingual work, it's the strongest open model on European languages — French, German, Spanish, Italian, Portuguese all scoring above 81% on multilingual MMLU, the result of deliberate data curation rather than incidental coverage. The MIT license makes it genuinely deployable without legal review overhead, which matters for the enterprise customers Mistral was targeting. Large 2 was a statement: a European lab with no hyperscaler backing could ship a model that competed credibly with OpenAI and Anthropic on the benchmarks developers actually run before choosing a provider. At 123B, it was also large enough that you needed real infrastructure to serve it — not a laptop model — but small enough to fit in a single high-memory node.

Black Forest Labs — founded by the core team behind Stable Diffusion, including Robin Rombach — releases Flux.1 on August 1 across three tiers: Flux.1 Schnell (Apache 2.0, fastest inference), Flux.1 Dev (non-commercial research), and Flux.1 Pro (commercial API). The benchmark that circulated immediately: Flux consistently outperformed SDXL, DALL-E 3, and Midjourney v6 on compositional accuracy, anatomy, and — most visibly — text rendering. Legible text inside generated images had eluded every prior model; Flux did it reliably. Within a week, every major ComfyUI workflow and image-generation pipeline had native Flux support. The Apache 2.0 Schnell variant became the fastest-adopted base model in the open-weights image ecosystem. Flux marked the culmination of the Stability AI talent exodus: the people who built Stable Diffusion had now shipped its replacement, on their own terms, with better results.

OpenAI ships o1-preview and o1-mini: models that 'think before they answer' via long internal chain-of-thought reasoning. o1-preview hits PhD-level performance on science benchmarks. The trade: slower, more expensive, less fluent prose. But for hard math, code, and logic problems, it was visibly different. A new capability tier, not just an incremental update.

Alibaba releases Qwen 2.5 across dense and MoE variants, describing it as 'perhaps the largest open-source release in history.' Ships alongside Qwen2.5-Coder and Qwen2.5-Math, each trained on domain-specific corpora. The family demonstrated that systematic specialization — separate model families for distinct problem types — could yield gains that generalist training couldn't match.

Google ships Audio Overviews in NotebookLM — converts any uploaded document into a two-host podcast. The demo circulated instantly: AI hosts debating and explaining research papers with natural banter, interruptions, and laughter. Demonstrated that synthetic audio had crossed a quality threshold where most listeners couldn't distinguish it from human production. The podcast format became a new interface for consuming dense material.

Meta releases Llama 3.2 in four sizes: 1B and 3B text-only models optimized for edge and mobile, plus 11B and 90B vision models. The 1B and 3B variants are the first serious on-device open models — 128K context, fast, and small enough to ship in an app. The 90B vision model surpasses Claude 3 Haiku on image understanding.

OpenAI ships GPT-4 Omni: natively multimodal across text, audio, image, and video — no separate encoders, one unified model. Response latency of 232ms matched human conversational speed. The live demo showed it singing, responding to facial expressions, flirting. The 'Her' moment. Made free to all ChatGPT users on day one, which is how it got 100M people testing omni-modal AI in a week.

Anthropic releases Claude 3.5 Sonnet on June 20, 2024 — at 80% lower cost than Claude 3 Opus, while outperforming it on most benchmarks. The initial SWE-bench Verified score is 33.4%, which already leads all publicly available models at launch and clears GPT-4o (which sat meaningfully lower on the same benchmark). An October update to the same model family later pushes this to 49.0%, the state of the art at the time. GPQA graduate-level reasoning hits 59.4% versus 53.6% for GPT-4o — the clearest sign that Sonnet-class Claude had moved past GPT-4o on the tasks where reasoning depth matters most. Ships with Artifacts: a persistent side-panel for code Claude generates, making iterative development feel more like pair programming than chat. The 200K context window was double GPT-4o's 128K, which mattered for codebase-scale work immediately. This is the release where Claude stops being 'the Anthropic model' developers occasionally reach for and starts being the daily driver for a significant slice of production software development.

Meta releases Llama 3 in 8B and 70B sizes, trained on 15 trillion tokens — 7x more data than Llama 2. The 70B model rivals GPT-3.5 on most benchmarks and exceeds Llama 2 70B on everything. Open weights, commercial-friendly license, and immediate integration into Ollama, LM Studio, and the broader local inference ecosystem.