A little-known Miami-based startup called Subquadratic emerged from stealth on Tuesday with a sweeping claim: that it has built the first large language model to fully escape the mathematical constraint that has defined — and limited — every major AI system since 2017.
The company claims its first model, SubQ 1M-Preview, is the first LLM built on a fully subquadratic architecture — one where compute grows linearly with context length. If that claim holds, it would be a genuine inflection point in how AI systems scale. At 12 million tokens, the company says, its architecture reduces attention compute by almost 1,000 times compared to other frontier models — a figure that, if validated independently, would dwarf the efficiency gains of any existing approach.
The company is also launching three products into private beta: an API exposing the full context window, a command-line coding agent called SubQ Code, and a search tool called SubQ Search. It has raised $29 million in seed funding from investors including Tinder co-founder Justin Mateen, former SoftBank Vision Fund partner Javier Villamizar, and early investors in Anthropic, OpenAI, Stripe, and Brex. The New Stack reported that the raise values the company at $500 million.
The numbers Subquadratic is publishing are extraordinary. The reaction from the AI research community has been, to put it mildly, mixed — ranging from genuine curiosity to open accusations of vaporware. Understanding why requires understanding what the company claims to have solved, and why so many prior attempts to solve the same problem have fallen short.
Every transformer-based AI model — which includes virtually every frontier system from OpenAI, Anthropic, Google, and others — relies on an operation called “attention.” Every token is compared against every other token, so as inputs grow, the number of interactions — and the compute required to process them — scales quadratically. In plain terms: double the input size, and the cost doesn’t double. It quadruples.
This relationship has shaped what gets built and what doesn’t. The industry standard is 128,000 tokens for many AI models and up to 1 million tokens for frontier cloud models such as Claude Sonnet 4.7 and Gemini 3.1 Pro.
Even at those sizes, the cost of processing long inputs becomes punishing. The industry built an elaborate stack of workarounds to cope. RAG systems use a search engine to pull a small number of relevant results before sending them to the model, because sending the full corpus isn’t feasible. Developers layer retrieval pipelines, chunking strategies, prompt engineering techniques, and multi-agent orchestration systems on top of models — all to route around the fundamental constraint that the model itself can’t efficiently process everything at once.
Subquadratic’s argument is that these workarounds are expensive, brittle, and ultimately limiting. As CTO Alexander Whedon told SiliconANGLE in an interview, “I used to manually curate prompts and retrieval systems and evals and conditional logic to chain together the workflows. And I think that that is kind of a waste of human intelligence and also limiting to the product quality.”
The company’s approach, called Subquadratic Sparse Attention or SSA, is built on a straightforward premise: most of the token-to-token comparisons in standard attention are wasted compute. Instead of comparing every token to every other token, SSA learns to identify which comparisons actually matter and computes attention only over those positions. Crucially, the selection is content-dependent — the model decides where to look based on meaning, not on fixed positional patterns. This allows it to retrieve specific information from arbitrary positions across a very long context without paying the quadratic tax.
The practical payoff scales with context length — exactly the inverse of the problem it’s trying to solve. According to the company’s technical blog, SSA achieves a 7.2x prefill speedup over dense attention at 128,000 tokens, rising to 52.2x at 1 million tokens. As Whedon put it: “If you double the input size with quadratic scaling laws, you need four times the compute; with linear scaling laws, you need just twice.” The company says it trained the model in three stages — pretraining, supervised fine-tuning, and a reinforcement learning stage specifically targeting long-context retrieval failures — teaching the model to aggressively use distant context rather than defaulting to nearby information, a subtle failure mode that quietly degrades performance in existing systems.
On the surface, SubQ’s benchmark numbers are competitive with or superior to models built by organizations spending billions of dollars. On SWE-Bench Verified, it scored 81.8% compared to Opus 4.6’s 80.8% and DeepSeek 4.0 Pro’s 80.0%. On RULER at 128,000 tokens, a standard benchmark for reasoning over extended inputs, SubQ scored 95% — edging out Claude Opus 4.6 at 94.8%. On MRCR v2, a demanding test of multi-hop retrieval across long contexts, SubQ posted a third-party verified score of 65.9%, compared with Claude Opus 4.7 at 32.2%, GPT-5.5 at 74%, and Gemini 3.1 Pro at 26.3%.
But several details warrant scrutiny. The benchmark selection is narrow — exactly three tests, all emphasizing long-context retrieval and coding, the precise tasks SubQ is designed for. Broader evaluations across general reasoning, math, multilingual performance, and safety have not been published. The company says a comprehensive model card is “coming soon.”
According to The New Stack, each benchmark model was run only once due to high inference cost, and the SWE-Bench margin is, as the company’s own paper acknowledges, “harness as much as model.” In benchmark methodology, single runs without confidence intervals leave room for variance. There is also a significant gap between SubQ’s research results and its production model. On MRCR v2, the company reported a research score of 83 — but the third-party verified production model scored 65.9. That 17-point gap between the lab result and the shipping product is notable and largely unexplained.
Subquadratic also told SiliconANGLE that on the RULER 128K benchmark, SubQ scored 95% accuracy at a cost of $8, compared with 94% accuracy and about $2,600 for Claude Opus — a remarkable cost claim. But the company has not publicly disclosed specific API pricing, making it impossible to independently verify the cost-per-task comparisons.
Within hours of the announcement, the AI research community erupted into a debate that crystallized around a single question: Is this real?
AI commentator Dan McAteer captured the binary mood in a widely shared post: “SubQ is either the biggest breakthrough since the Transformer… or it’s AI Theranos.” The comparison to the infamous blood-testing fraud company may be unfair, but it reflects the scale of the claims being made. Skeptics zeroed in on several pressure points. Prominent AI engineer Will Depue initially noted that SubQ is “almost surely a sparse attention finetune of Kimi or DeepSeek,” referring to existing open-source models.
Whedon confirmed this on X, writing that the company is “using weights from open-source models as a starting point, as a function of our funding and maturity as a company.” Depue later escalated his criticism, writing that the company’s O(n) scaling claims and the speedup numbers “don’t seem to line up” and called the communication “either incredibly poorly communicated or just not real.”
Others raised structural questions. One developer noted that if SubQ truly reduces compute by 1,000x and costs less than 5% of Opus, the company should have no trouble serving it at scale — so why gate access through an early-access program? Developer Stepan Goncharov called the benchmarks “very interesting cherry-picked benchmarks,” while another commenter described them as “suspiciously perfect.”
But not everyone was dismissive. AI researcher John Rysana pushed back on the Theranos framing, writing that the work is “just subquadratic attention done well which is very meaningful for long context workloads,” and that “odds of it being BS are extremely low.” Linus Ekenstam, a tech commentator, said he was “extremely intrigued to see the real-world implications” particularly for complex AI-powered software.
Perhaps the most pointed critique of SubQ’s launch comes not from its specific claims but from recent history. Magic.dev announced a 100-million-token context-window model in August 2024, with a claimed 1,000x efficiency advantage, and raised roughly $500 million on the strength of those claims. As of early 2026, there is no public evidence of LTM-2-mini being used outside Magic.
The parallels are uncomfortable. Both companies claimed massive context windows. Both touted roughly 1,000x efficiency gains. Both targeted software engineering as their primary use case. And both launched with limited external access.
The broader research landscape reinforces the caution. Kimi Linear, DeepSeek Sparse Attention, Mamba, and RWKV all promised subquadratic scaling, and all faced the same problem: architectures that achieve linear complexity in theory often underperform quadratic attention on downstream benchmarks at frontier scale, or they end up hybrid — mixing subquadratic layers with standard attention and losing the pure scaling benefits.
A widely cited LessWrong analysis argued that these approaches “are all better thought of as ‘incremental improvement number 93595 to the transformer architecture'” because practical implementations remain quadratic and “only improve attention by a constant factor.”
Subquadratic is directly aware of this history. Its own technical blog specifically addresses each prior approach — fixed-pattern sparse attention, state space models, hybrid architectures, and DeepSeek Sparse Attention — and argues that SSA avoids their tradeoffs. Whether it actually does remains an empirical question that only independent evaluation can settle.
The team behind the claims matters in evaluating them. CEO Justin Dangel is a five-time founder and CEO with a track record across health tech, insurancetech, and consumer goods, and his companies have scaled to hundreds of employees, attracted institutional backing, and reached liquidity. CTO Alexander Whedon previously worked as a software engineer at Meta and served as Head of Generative AI at TribeAI, where he led over 40 enterprise AI implementations.
The team includes 11 PhD researchers with backgrounds from Meta, Google, Oxford, Cambridge, ByteDance, and Adobe. That is a credible collection of talent for an architecture-level research effort. But neither co-founder has published foundational AI research, and the company has not yet released a peer-reviewed paper. The technical report is listed as “coming soon.”
The funding profile is unusual for a company making frontier AI claims. Subquadratic raised $29 million at a reported $500 million valuation — a steep price for a seed-stage company with no publicly available model, no peer-reviewed research, and no disclosed revenue. The investor base, led by Tinder co-founder Mateen and former SoftBank partner Villamizar, skews toward consumer tech and growth investing rather than deep technical AI research. The company is not open-sourcing its weights but plans to offer training tools for enterprises to do their own post-training, and has set a 50-million-token context window target for Q4.
Strip away the marketing language and the social media drama, and the underlying question Subquadratic is asking is genuinely important: Can AI systems break free of quadratic scaling without sacrificing the quality that makes them useful?
The stakes are enormous. If attention can be made truly linear without degrading retrieval and reasoning, the economics of AI shift fundamentally. Enterprise applications that today require elaborate retrieval pipelines — processing entire codebases, contracts, regulatory filings, medical records — become single-pass operations. The billions of dollars currently spent on RAG infrastructure, context management, and agentic orchestration become partially redundant.
Whedon’s willingness to engage publicly with technical criticism — posting a technical blog within hours of pushback — suggests a team that understands it needs to show its work, not just describe it. And to its credit, the company acknowledged openly that it builds on open-source foundations and that its model is smaller than those at the major labs.
Every frontier model in 2026 advertises a context window of at least a million tokens, but almost none of them are actually great at making use of all that information. The gap between a nominal context window and a functional one — between what a model accepts and what it reliably reasons over — remains one of the most important unsolved problems in AI. Subquadratic says it has closed that gap. If independent evaluation confirms that claim, the implications would ripple far beyond a single startup’s valuation. If it doesn’t, the company joins a growing list of long-context promises that sounded revolutionary on launch day and unremarkable six months later.
In computing, every fundamental constraint eventually falls. When it does, the breakthrough never comes from the direction the industry expected. The question hanging over Subquadratic is whether a team of 11 PhDs and a $29 million seed round actually found the answer that has eluded organizations spending thousands of times more — or whether they just found a better way to describe the problem.
The scaffolding layer that developers once needed to ship LLM applications — indexing layers, query engines, retrieval pipelines, carefully orchestrated agent loops — is collapsing. And according to Jerry Liu, co-founder and CEO of LlamaIndex, that’s not a problem. It’s the point.
“As a result, there’s less of a need for frameworks to actually help users compose these deterministic workflows in a light and shallow manner,” Jerry Liu, co-founder and CEO of LlamaIndex, explains in a new VentureBeat Beyond the Pilot podcast.
Liu’s LlamaIndex is one of the foremost retrieval-augmented generation (RAG) frameworks connecting private, custom, and domain-specific data to LLMs. But even he acknowledges that these types of frameworks are becoming less relevant.
With every new release, models demonstrate incremental capabilities to reason over “massive amounts” of unstructured data, and they’re getting better at it than humans, he notes. They can be trusted to reason extensively, self-correct, and perform multi-step planning; Modern Context Protocol (MCP) and Claude Agent Skills plug-ins allow models to discover and use tools without requiring integrations for every one independently.
Agent patterns have consolidated toward what Liu calls a “managed agent diagram” — a harness layer combined with tools, MCP connectors, and skills plug-ins, rather than custom-built orchestration for every workflow.
Further, coding agents excel at writing code, meaning devs don’t need to rely on extensive libraries. In fact, about 95% of LlamaIndex code is generated by AI. “Engineers are not actually writing real code,” Liu said. “They’re all typing in natural language.” This means the layers between programmers and non-programmers is collapsing, because “the new programming language is essentially English.”
Instead of manual coding or struggling to understand API and document integration, devs can just point Claude Code at it. “This type of stuff was either extremely inefficient or just would break the agent three years ago,” said Liu. “It’s just way easier for people to build even relatively advanced retrieval with extremely simple primitives.”
So what’s the core differentiator when the stack collapses?
Context, Liu says. Agents need to be able to decipher file formats to extract the right information. Providing higher accuracy and cheaper parsing becomes key, and LlamaIndex is well-positioned here, he contends, because of its developments with agentic document processing via optical character recognition (OCR).
“We’ve really identified that there’s a core set of data that has been locked up in all these file format containers,” he said. Ultimately, “whether you use OpenAI Codex or Claude Code doesn’t really matter. The thing that they all need is context.”
There’s growing concern about builders like Anthropic locking in session data; in light of this, Liu emphasizes the importance of modularity and agnosticism. Builders shouldn’t bet on any one frontier model, or overbuild in a way that overcomplicates components of the stack.
Retrieval has evolved into “agent-plus-sandbox,” as he describes it, and enterprises must ensure that their code bases are tech debt free and adaptable to changing patterns. They also have to acknowledge that some parts of the stack will eventually need to be thrown away as a matter of course.
“Because with every new model release, there’s always a different model that is kind of the winner,” Liu said. “You want to make sure you actually have some flexibility to take advantage of it.”
Listen to the podcast to hear more about:
LlamaIndex’s beginnings as a ‘toy project’ with initially only about 40% accuracy;
How SaaS companies can tap into complicated workflows that must be standardized and repeatable for average knowledge workers;
Why vertical AI companies are taking off and why ‘build versus buy’ is still a very valid question in the agent age.
You can also listen and subscribe to Beyond the Pilot on Spotify, Apple or wherever you get your podcasts.
Runpod, the high-performance cloud computing and GPU platform designed specifically for AI development, today launched a new open source, MIT licensed, enterprise-friendly Python programming tool called Runpod Flash — and it is poised to make creation, iteration and deployment of AI systems inside and outside of foundation model labs much faster.
The tool aims to eliminate some of the biggest barriers and hurdles to training and using AI models today, namely, doing away with Docker packages and containerization when developing for serverless GPU infrastructure, which the company believes will speed up development and deployment of new AI models, applications and agentic workflows.
Additionally, the platform is built to serve as a critical substrate for AI agents and coding assistants—such as Claude Code, Cursor, and Cline—enabling them to orchestrate and deploy remote hardware autonomously with minimal friction.
Developers can utilize Flash to accomplish a diverse set of high-performance computing tasks, including cutting-edge deep learning research, model training, and fine-tuning.
“We make it as easy as possible to be able to bring together the cosmos of different AI tooling that’s available in a function call,” said RunPod chief technology officer (CTO) Brennen Smith, in a video call interview with VentureBeat last week.
The tool allows for the creation of sophisticated “polyglot” pipelines, where users can route data preprocessing to cost-effective CPU workers before automatically handing off the workload to high-end GPUs for inference.
Beyond research and development, Flash supports production-grade requirements through features such as low-latency load-balanced HTTP APIs, queue-based batch processing, and persistent multi-datacenter storage.
The core value proposition of Flash GA is the removal of Docker from the serverless development cycle.
In traditional serverless GPU environments, a developer must containerize their code, manage a Dockerfile, build the image, and push it to a registry before a single line of logic can execute on a remote GPU. Runpod Flash treats this entire process as a “packaging tax” that slows down iteration cycles.
Under the hood, Flash utilizes a cross-platform build engine that enables a developer working on an M-series Mac to produce a Linux x86_64 artifact automatically.
This system identifies the local Python version, enforces binary wheels, and bundles dependencies into a deployable artifact that is mounted at runtime on Runpod’s serverless fleet.
This mounting strategy significantly reduces “cold starts”—the delay between a request and the execution of code—by avoiding the overhead of pulling and initializing massive container images for every deployment.
Furthermore, the technology infrastructure supporting Flash is built on a proprietary Software Defined Networking (SDN) and Content Delivery Network (CDN) stack.
Smith told VentureBeat that the hardest problems in GPU infrastructure are often not the GPUs themselves, but the networking and storage components that link them together.
“Everyone is talking about agentic AI, but the way I personally see it — and the way the leadership team at RunPod sees it — is that there needs to be a really good substrate and glue for these agents, whatever they might be powered by, to be able to work with,” Smith said.
Flash leverages this low-latency substrate to handle service discovery and routing, enabling cross-endpoint function calls. This allows developers to build “polyglot” pipelines where, for instance, a cheap CPU endpoint handles data preprocessing before routing the clean data to a high-end NVIDIA H100 or B200 GPU for inference.
While the Flash beta focused on live-test endpoints, the GA release introduces a suite of features designed for production-grade reliability.
The primary interface is the new @Endpoint decorator, which consolidates configuration—such as GPU type, worker scaling, and dependencies—directly into the code. The GA release defines four distinct architectural patterns for serverless workloads:
Queue-based: Designed for asynchronous batch jobs where functions are decorated and run.
Load-balanced: Tailored for low-latency HTTP APIs where multiple routes share a pool of workers without queue overhead.
Custom Docker Images: A fallback for complex environments like vLLM or ComfyUI where a pre-built worker is already available.
Existing Endpoints: Using Flash as a Python client to interact with previously deployed Runpod resources via their unique IDs.
A critical addition for production environments is the NetworkVolume object, which provides first-class support for persistent storage across multiple datacenters.
Files mounted at /runpod-volume/ allow for model weights and large datasets to be cached once and reused, further mitigating the impact of cold starts during scaling events.
Additionally, Runpod has introduced environment variable management that is excluded from the configuration hash, meaning developers can rotate API keys or toggle feature flags without triggering an entire endpoint rebuild.
To address the rise of AI-assisted development, Runpod has released specific skill packages for coding agents like Claude Code, Cursor, and Cline.
These packages provide agents with deep context regarding the Flash SDK, effectively reducing syntax hallucinations and allowing agents to write functional deployment code autonomously.
This move positions Flash not just as a tool for humans, but as the “substrate and glue” for the next generation of AI agents.
Runpod has released the Flash SDK under the MIT License, one of the most permissive open-source licenses available.
This choice is a deliberate strategic move to maximize market share and developer adoption. In contrast to more restrictive licenses like the GPL (General Public License), which can impose “copyleft” requirements—potentially forcing companies to open-source their own proprietary code if it links to the library—the MIT license allows for unrestricted commercial use, modification, and distribution.
Smith explained this philosophy as a “motivating construct” for the company: “I prefer to win based on product quality and product innovation rather than legal ease and lawyers,” he told VentureBeat.
By adopting a permissive license, Runpod lowers the barrier for enterprise adoption, as legal teams do not have to navigate the complexities of restrictive open-source compliance.
Furthermore, it invites the community to fork and improve the tool, which Runpod can then integrate back into the official release, fostering a collaborative ecosystem that accelerates the development of the platform.
The launch of Flash GA comes at a time of explosive growth for Runpod, which has surpassed $120 million in Annual Recurring Revenue (ARR) and serves a developer base of over 750,000 since it was founded in 2022.
The company’s growth is driven by two distinct segments: the “P90” enterprises—large-scale operations like Anthropic, OpenAI, and Perplexity—and the “sub-P90” independent researchers and students who represent the vast majority of the user base.
The platform’s agility was recently demonstrated during the release of DeepSeek V4 in preview last week. Within minutes of the model’s debut, developers were utilizing Runpod infrastructure to deploy and test the new architecture.
This “real-time” capability is a direct result of Runpod’s specialized focus on AI developers, offering over 30 GPU SKUs and billing by the millisecond to ensure that every dollar of spend results in maximum throughput.
Runpod’s position as the “most cited AI cloud on GitHub” suggests that it has successfully captured the developer mindshare required to sustain its momentum.
With Flash GA, the company is attempting to transition from being a provider of raw compute to becoming the essential orchestration layer for the AI-first cloud.
As development shifts toward “intent-based” coding—where the outcome is prioritized over the execution details—tools that bridge the gap between local ideas and global scale will likely define the next era of computing.
Amazon Web Services on Tuesday launched one of the most consequential enterprise AI plays in the company’s 20-year history, simultaneously bringing OpenAI’s most powerful models to its Bedrock platform, unveiling a new agentic developer framework, releasing a desktop AI productivity tool called Amazon Quick, and expanding its Amazon Connect service from a single contact-center product into a family of four agentic AI solutions targeting supply chains, hiring, healthcare, and customer experience.
The announcements, made at a live event in San Francisco titled “What’s Next with AWS,” landed just 24 hours after OpenAI and Microsoft publicly restructured their exclusive cloud partnership — a move that, for the first time, freed OpenAI to distribute all of its products across rival cloud providers. AWS CEO Matt Garman called it “a huge partnership” and said customers have been asking for OpenAI models inside AWS “from the very early days.”
The timing was no accident. Amazon CEO Andy Jassy had flagged the Microsoft-OpenAI restructuring as “very interesting” in a post on X the day prior, promising more details on Tuesday. What followed was a sweeping set of launches that together represent AWS’s bid to become the definitive infrastructure layer for the agentic AI era — one where intelligent software agents don’t just answer questions but take autonomous action inside enterprise workflows.
The centerpiece announcement: OpenAI’s latest models are now available through Amazon Bedrock in limited preview, with general availability expected within weeks. AWS confirmed that GPT-5.4 is available immediately in limited preview, with GPT-5.5 arriving shortly thereafter.
In an exclusive interview with VentureBeat at the event, Anthony Liguori, Vice President and Distinguished Engineer at AWS, described the significance of the moment. “We announced a partnership about eight weeks ago centered around this idea of the stateful runtime environment, the SRE APIs,” Liguori said. “However, today we announced the availability of all of OpenAI’s frontier models in Amazon Bedrock available via both the stateless APIs — these are the APIs that are commonly used, like chat completions and responses.”
Liguori characterized the stateless API availability as particularly critical because it removes migration friction. “Customers can take their existing workloads today and just start using AWS right off the bat,” he said. “They don’t have to write any new software, develop any new things. I think that’s one of the most exciting announcements that came out today.”
The integration means AWS customers can now evaluate and deploy OpenAI models alongside offerings from Anthropic, Meta, Mistral, Cohere, and Amazon’s own models — all through Bedrock’s unified security, governance, and cost controls. For enterprise procurement teams, this collapses what had been a fragmented multi-vendor landscape into a single pane of glass.
The path to Tuesday’s announcement was anything but smooth. As TechCrunch reported, OpenAI’s earlier $50 billion deal with Amazon, announced in February, had created a legal tangle with Microsoft. Under the original Microsoft-OpenAI agreement, Microsoft retained exclusive rights to OpenAI products accessed through APIs, which appeared to conflict directly with OpenAI’s promise to give AWS exclusive hosting rights for its new Frontier agent-building tool.
Microsoft had publicly pushed back at the time, stating that “Azure remains the exclusive cloud provider of stateless OpenAI APIs.” The Financial Times reported that Microsoft even contemplated legal action. Monday’s restructured deal — which replaced Microsoft’s open-ended exclusivity with a nonexclusive license running through 2032 — swept those legal obstacles aside.
For AWS, the resolution means its multi-billion-dollar investment in OpenAI can now fully bear fruit. As CNBC reported, OpenAI’s revenue chief Denise Dresser had told employees in a memo that the Microsoft relationship “has also limited our ability to meet enterprises where they are — for many that’s Bedrock.” At the San Francisco event, Dresser framed the moment as a turning point. “They’re no longer in the mindset of experimentation and pilots,” she said of enterprise customers. “They really want to go full enterprise wide, and they understand that to do that, they need to have powerful models. But even more importantly, they want those models in a trusted environment.”
OpenAI CEO Sam Altman, who was unable to attend in person due to his ongoing court case against Elon Musk across the Bay Bridge in Oakland, sent a recorded video message. “We are co-developing an agent platform from the ground up, deeply integrated with AWS services and powered by OpenAI’s most advanced models and tools,” Altman said, “so that customers can build and run powerful agents in their own environment without worrying about the underlying plumbing.”
Beyond raw model access, AWS launched Amazon Bedrock Managed Agents powered by OpenAI — a system that combines OpenAI’s frontier models with its proprietary “harness,” the agentic execution framework that powers products like Codex. This is where Liguori’s technical analysis was most revealing.
He explained that the harness concept represents a shift in how models are trained and deployed for agentic work. “When you think about an agentic platform, there’s really two components,” Liguori told VentureBeat. “One is the harness — the actual logic that will execute tool calls for the model, determine when to compact the context, all of those sorts of things — and then the model itself.”
Critically, Liguori argued, the best agentic performance comes when models are trained specifically against their harness through reinforcement learning — not merely prompted to use tools at inference time. “You can give a model a whole lot of instructions and a set of tools, and it will be able to use it most of the time,” he said. “But when you really train the model on a specific set of tools, a specific style of operations, it’s just like drilling plays over and over again — the model builds muscle memory for using that harness.”
The football analogy is instructive. Where general-purpose models are like versatile athletes who can adapt to any playbook, harness-trained models are like championship teams that have run the same formations thousands of times until execution becomes instinctive. For enterprises deploying agents in high-stakes production environments — managing financial transactions, orchestrating supply chains, or processing sensitive healthcare data — that reliability gap matters enormously.
Bedrock Managed Agents consists of three components: a runtime layer for configuring skills, memory policies, and tool access; an environment layer where the agent lives (deployable on Fargate or other AWS compute); and an inference API for interacting with the agent. The system integrates deeply with AWS’s identity and access management, VPC networking, and CloudTrail auditing — meaning every action an agent takes is logged and governed by existing enterprise security policies.
Liguori made what may be his most striking claim when discussing why enterprises should trust AWS over on-premises alternatives or smaller cloud providers. “With Bedrock, the system that we’re using to host the GPT-5.4 models, that whole environment is zero operator access,” he told VentureBeat. “There’s no human that could ever log into one of those machines, so your inference data is never able to be accessed by a human.”
He pointed to AWS’s custom silicon — Graviton processors and Nitro security chips — as the foundation for this claim. “When you look at one of our servers, either compute servers or the servers we’re using for Gen AI, the only thing that you can buy off the shelf is the memory modules. Everything else is either custom boards or even custom silicon.”
This argument is designed to counter a growing narrative from what the industry calls “neo-clouds” — smaller providers that offer on-premises model hosting with tighter physical security controls. Liguori flipped that argument on its head: “You’re actually way more secure in the cloud because we have built a platform with such strong physical securities… If you were to try to stand up your own inference system today, you’d probably be running open source software on just Linux.”
It’s a bold claim, and one that enterprise CISOs will undoubtedly scrutinize. But it underscores AWS’s conviction that the agentic era — where AI agents access source code, PII data, and critical business systems — demands infrastructure security guarantees that go far beyond what most organizations can build independently.
OpenAI’s Codex coding agent also arrived on Bedrock in limited preview. Dresser shared that Codex has been growing at a blistering pace, expanding “from 3 million weekly active users to 4 million in two weeks.” The tool has evolved beyond simple code generation into a full agentic software development lifecycle platform.
For Liguori, who described himself as “10 to 20 times more productive” as an engineer thanks to tools like Codex, bringing this capability into AWS represents the bridge between individual developer productivity and enterprise-scale deployment. “Most developers today are using these OpenAI models on their laptops,” he said. “We haven’t seen that happen yet in the rest of the industry, and with Bedrock Managed Agents, we think we have a way for enterprises to deploy agents in a means that meets their compliance requirements.”
The gap Liguori is describing — between the solo developer experience and enterprise-wide adoption — is arguably the central challenge of the current AI moment. Individual engineers can achieve extraordinary productivity gains with agentic coding tools. But scaling that to thousands of developers across a Fortune 500 company, with proper governance, security, and auditability, requires platform-level infrastructure. That’s the market AWS is targeting.
Liguori saw the near-term potential in even more immediate terms. He described leading a team of about 20 engineers who share a common codebase of skills and MCP tools. “That has been an amazingly powerful thing, because we’re all able to build on top of each other as we learn how to use these models,” he said. “Where I’ve run into a hurdle is there’s a lot of stuff I’d like to share with our finance team… and I can’t really ask them to clone a Git repo and build it from a Git repo.” Bedrock Managed Agents, he argued, will let teams create hosted agents that non-technical colleagues can access — taking agentic development from a developer-only practice to an enterprise-wide capability within the next six months.
While the OpenAI partnership dominated headlines, AWS also launched Amazon Quick Desktop — a new desktop application designed to bring agentic AI to knowledge workers who aren’t developers. Liguori framed the product as addressing a critical gap. “A lot of these agentic tools have primarily targeted developers,” he said. “Quick Desktop is a really great tool if you are a knowledge worker that is not a developer… I think it’s been underserved for the non-developer knowledge workers.”
Quick Desktop integrates with a user’s local files, calendar, email, Slack, and enterprise applications — building what AWS calls a “Knowledge Graph” that maps relationships between people, projects, decisions, and actions. The system connects natively with Google Workspace, Microsoft 365, Zoom, and Salesforce. Unlike other AI productivity tools, Quick doesn’t wait for prompts. It proactively surfaces what matters — unanswered emails, deals needing updates, documents awaiting review — and can take action like scheduling meetings, drafting emails, or updating Jira tickets.
Garman, who said he had been using the desktop app for several weeks, called it “by far the most effective tool” among AI productivity products he has tested. “If you think about what we’ve done with Quick — combine all of your sources of data inside of the enterprise — but then we also saw the power of having access to a local desktop and being able to operate with your local files and your local email and your local Slack… but people were worried about security, appropriately so,” Garman said. “What we’re doing here is combining a bunch of those things together with QUIC to give you the best of all of those worlds.”
The product is available in preview today, with no AWS account required — users can sign up with just an email address. Customers including BMW, 3M, Mondelēz, Southwest Airlines, and the NFL are already using it, with some reporting production time reductions of nearly 80% and customer issue processing cut by more than 50%.
Perhaps the most ambitious long-term bet announced Tuesday was the expansion of Amazon Connect from a single contact-center product — one that reached over $1 billion in revenue last year and processes 20 million interactions daily — into a family of four agentic AI solutions.
The new lineup includes Amazon Connect Decisions, an agentic supply chain planning tool built on more than 25 specialized supply chain tools and 30 years of Amazon operational science, including one of Amazon’s SCOT (Supply Chain Optimization Technologies) foundation models. Amazon Connect Talent is a high-volume hiring platform inspired by Amazon’s experience hiring 250,000 seasonal employees during peak periods, using AI agents to conduct voice interviews around the clock and present recruiters with anonymized, skills-based scoring. Amazon Connect Customer AI is the renamed and enhanced version of the original contact-center service. And Amazon Connect Health covers the patient journey from appointment scheduling through clinical encounters, including ambient documentation, billing code suggestions, and post-visit summaries drawn from Amazon’s experience with One Medical and Amazon Pharmacy.
Colleen Aubrey, who leads applied AI solutions at AWS and previously co-founded Amazon’s advertising business, introduced a new design philosophy underlying all four products: “humorphism.” Where skeuomorphism translated physical objects into digital metaphors — desks to desktops, files to folders — humorphism translates human interaction dynamics into AI agent behavior. “If we’re building products that at the heart of which is an agentic teammate, then how should those teammates interact with you?” Aubrey asked. The philosophy manifests in specific design choices: Connect Decisions agents ask planners why they made manual adjustments and apply those insights across similar products. Connect Talent agents adapt follow-up questions based on candidate responses. Connect Health agents trace every clinical insight back to source data so physicians can verify AI-generated documentation.
Taken together, Tuesday’s announcements reveal a coherent strategy operating across four distinct layers: custom infrastructure (Graviton, Trainium, zero-operator-access security), model access (Bedrock as a model marketplace with unified APIs), an agentic platform (Bedrock Managed Agents and AgentCore for building and governing agents), and purpose-built applications (Quick for individual productivity, Connect for vertical business operations).
This layered approach addresses a fundamental tension in the enterprise AI market. Companies want choice at the model layer but integration at the platform layer and specificity at the application layer. By offering all three through a single security and governance framework, AWS is betting it can capture value across the entire stack — a strategy that reshapes competitive dynamics for Microsoft, Google Cloud, and the growing constellation of smaller AI infrastructure providers.
Garman pushed back on the “SaaSpocalypse” narrative that agentic AI will destroy incumbent enterprise software companies. “The incumbent providers today have such a huge advantage,” he said. “They have deep domain expertise… a large customer set with all of their data.” He pointed to Salesforce’s recent headless API offering as an example of incumbents adapting smartly. But he also drew an explicit parallel to the early days of cloud computing, when customers would simply replicate their on-premises data centers in the cloud rather than reimagine what was possible. “You see that today with how people are thinking about AI and agents,” Garman said. “They’re like, ‘I have this business process, I’m gonna have agents do the exact same thing that humans do.’ It kind of works… but it doesn’t give you that transformational change.”
He pointed to Amazon’s own Prime Video team as proof of what that change looks like in practice. The team used agentic tools to rebuild a partner payment system that was projected to take two years — completing it in roughly two quarters with a handful of people, while simultaneously improving the system for customers, for Amazon, and for the partners who get paid through it.
For enterprises evaluating their AI strategies, Tuesday’s announcements simplify one decision — OpenAI models are now available where most of them already run production workloads — while complicating another. With model access increasingly commoditized across cloud providers, the real differentiator becomes the platform layer: where agents are built, governed, deployed, and trusted to take consequential actions. That’s the battleground AWS is staking out, and it’s the same ground Microsoft, Google, Salesforce, and a growing number of startups intend to contest.
Liguori sees the transformation accelerating fast. “I think what we’re going to see in the next six months is a lot of this agentic stuff going from developer only to being able to be consumed by a larger number of folks within an enterprise,” he told VentureBeat. Anthony Liguori, the AWS distinguished engineer who led the technical work over eight sleepless weeks to bring OpenAI’s models to Bedrock, said his own productivity as a software engineer has increased 10 to 20 times over the past year. When asked what excites him most about what comes next, he didn’t talk about models or infrastructure. He talked about what happens when that same multiplier reaches the finance team, the product managers, the supply chain planners — the millions of knowledge workers who have been watching the agentic revolution from the sidelines.
“We had nothing eight weeks ago,” he said, “and now we’re here.” If the next eight weeks move as fast, the sidelines may not exist for much longer.
Enterprises can’t fix their GPU waste problem because the fix makes the problem worse. Releasing idle capacity would improve utilization, but the same shortage driving GPU prices up is exactly why no team will give capacity back. So the fleet sits at roughly 5%, billed by the hour, and the cycle tightens.
That pressure — repeated across thousands of enterprises over the past two years — is the reason most companies are now running their GPU fleets at roughly 5% utilization, according to Cast AI’s 2026 State of Kubernetes Optimization Report, which measured actual production clusters rather than surveying them. It’s also the reason nobody releases the idle capacity. Cast AI co-founder and President Laurent Gil has been tracking the dynamic for two years. “Many of the neoclouds are not cloud,” he told VentureBeat. “They are neo-real estate.”
Five percent is about six times worse than a no-effort baseline. Gil puts a reasonable human-managed target at around 30% once you factor in day cycles, weekends and normal business patterns. Five percent means enterprises are running their most expensive infrastructure line at a fraction of what doing nothing intentional would yield. And it lands at the same moment cloud compute pricing has broken its 20-year pattern.
AWS quietly raised its reserved H200 GPU prices by roughly 15% on a Saturday in January, with no formal announcement. Memory suppliers pushed HBM3e prices up 20% for 2026. It is the first time since AWS launched EC2 in 2006 that a hyperscaler has meaningfully raised reserved GPU pricing rather than cut it. For now, the assumption under most enterprise AI budgets — that cloud compute gets cheaper every year— no longer holds at the top of the stack.
The pricing move matters less for what it is than for what it signals about where the shortage actually bites. Cloud compute has split into two layers. At the commodity layer, the old deflation still works. H100 on-demand pricing has fallen from roughly $7.57 per GPU-hour in September 2025 to around $3.93 today, with Lambda Labs and RunPod listing H100s under $3 and older A100s around $1.92. Nvidia T4 chips, once impossible to find on spot, now survive above 90% probability over 24 hours in several AWS regions.
At the frontier layer, it’s reversed. Nvidia received orders for 2 million H200 chips for 2026 against 700,000 in inventory. TSMC’s advanced packaging, which gates every HBM-equipped GPU, is booked through at least mid-2027. AMD has warned of its own 2026 price hikes citing the same crunch. Even A100 pricing, expected to soften as three-year reservations from 2023 expired, has started creeping back up. Gil’s read: FOMO is now spilling into older generations. Which layer an enterprise’s workloads sit on determines exposure.
How does fleet utilization get to 5% when GPUs are this expensive? Gil’s account of enterprise GPU procurement is the clearest explanation I have heard.
An enterprise needs GPUs. It joins a hyperscaler waitlist. Nothing happens for weeks, sometimes months. Then a phone call: “You asked for 48, I have 36. Yours if you want them, but only on a one-year or three-year commitment, and three years is cheaper. If you don’t want them, five other companies on the list will take them.” The fear of losing allocation is acute. The commitment gets signed. Whether the workloads will consume that many GPUs, or whether that chip generation fits what will run on them, is not the operative question at the moment. The operative question is whether to say yes or lose the slot.
Once secured, those GPUs become too painful to release. Reacquiring them would take months, and nobody wants to be the team that gave capacity back and couldn’t get it. So the fleet sits, billed by the hour, whether it is used or not. Gil described enterprises paying on-demand rates, roughly three times more expensive than one-year reservations, because even the premium felt safer than risking release.
This is the paradox at the center of the 5% number. The obvious way to improve utilization is to release the GPUs you are not using. But the very shortage that makes those GPUs expensive is also the reason nobody releases them. So the fleet stays over-provisioned, the shortage persists, prices rise, and the FOMO that started the cycle gets reinforced. Every turn of the loop makes the next exit harder.
Forrester’s data corroborates the dynamic from a different angle. Principal analyst Tracy Woo found practitioners self-estimating Kubernetes waste at around 60%, close to what Cast AI measures directly. A widely observed pattern in Kubernetes practice explains the dynamic: engineers routinely request five to ten times the resources they actually use, because the cost of under-provisioning is visible (a pager goes off) and the cost of over-provisioning is invisible (one line on a cloud bill no engineer sees).
Fixing procurement alone would not get the number to a good place, because the GPUs enterprises already hold are also wasteful on the inside. And the architecture half of the story is being diagnosed independently by teams that compete with Cast AI.
Anyscale, the company behind the Ray framework, published its own analysis on January 21 arguing that modern AI workloads routinely sit below 50% GPU utilization even when fleet size is exactly right, because of how the workloads are containerized. A single AI job moves through CPU-heavy stages (loading data, preprocessing), GPU-heavy stages (training or inference), and back to CPU. When all of that runs in one container, the GPU is allocated for the entire lifecycle but doing useful work for a fraction of it.
Gartner reaches the same conclusion independently. In a November 2025 research note on on-premises AI infrastructure, it recommends combining shared GPU usage across siloed projects with disaggregated inference, where prompt-processing and token-generation run on different hardware. Nvidia’s own Dynamo inference framework, unveiled for MLPerf Inference v6.0 last month, is built on the same principle.
Two vendors and an independent analyst firm (Cast AI, Anyscale, Gartner) converging on the same diagnosis is a stronger signal than any single vendor’s story, especially when one of them competes with the others. The two types of waste compound. A fleet over-committed at procurement time, running workloads whose containers leave GPUs idle waiting for CPU preprocessing, leaves enterprises at 5%. Fix one without fixing the other and most of the potential savings stay on the table.
If releasing GPUs is blocked by FOMO and procurement contracts are already signed, the only remaining lever is doing more useful work on the GPUs already committed. That is what “improve utilization” actually means in practice, and none of it requires buying a vendor’s product.
The simplest existence proof is the oldest technique in the book: GPU sharing across time zones. A bank with a credit decision engine serving Asian and US customers can run one pool of GPUs that serves both markets at different times. Nvidia published MIG (Multi-Instance GPU) and time-slicing primitives years ago. Most enterprises do not do it by hand because it is operationally boring and carries coordination overhead no one wants to own. An automated scheduler does it without getting tired.
Canva, the Australian design platform running over 100 production AI models, told Anyscale that it runs close to 100% GPU utilization during distributed training runs with roughly 50% cloud-cost reductions versus its previous setup. Inside Cast AI’s own data, a cluster of 136 H200 GPUs sustains 49% average utilization after applying GPU sharing, bin-packing (placing multiple workloads onto fewer, right-sized nodes), and a spot/on-demand mix. Ten times the fleet average and short of saturation, which is honest: most real enterprise fleets with mixed dev, staging, and production workloads probably sustain 40% to 70% at full optimization, not 100%. Even that is an order of magnitude better than 5%.
One caveat: the report’s 5% figure explicitly excludes AI labs running dedicated training. Organizations that look more like frontier labs than mixed enterprise fleets likely see much higher utilization already.
What should enterprises actually do differently in 2026? The paths available in the market are no longer interchangeable, and each makes a different bet on where supply and demand land.
|
Procurement path |
Typical H100-class price |
Availability |
Interruption risk |
Commitment |
Best fit |
|
Hyperscaler on-demand |
$3.00 to $6.98 per GPU-hour |
Limited for H100/H200 |
None |
None |
Unpredictable workloads, short runs |
|
Hyperscaler Capacity Blocks |
$4.33 to $4.97 per GPU-hour (H200 after Jan 2026) |
Pre-book up to 8 weeks; 6-month window |
None in window |
Medium-term |
Scheduled training with known windows |
|
Hyperscaler spot |
Up to 90% discount |
Variable; H100/H200 thin |
High (minutes of warning) |
None |
Fault-tolerant inference, checkpointed training |
|
Specialized GPU clouds (CoreWeave, Lambda, RunPod, GMI) |
$1.99 to $3.99 per GPU-hour for H100 |
Broader for newer generations |
Low to medium |
Per-run or short reservation |
Price-sensitive teams, flexible deployment |
|
On-premise or colocation |
Break-even around 12 to 18 months at sustained >60% utilization |
3 to 9 month lead times |
None |
3+ year capex |
High-utilization sustained workloads, strict compliance |
|
Decentralized marketplaces (Vast.ai, io.net, Aethir) |
Often under $1.00 per GPU-hour |
Highly variable quality |
High |
None |
Experimental or batch, non-production |
The pattern that no longer works is picking one path and locking in for a multi-year plan. A more defensible 2026 default is mixing paths against the split: commodity providers for workloads that can live there, hyperscaler Capacity Blocks only for workloads that need the guaranteed window.
None of the following requires buying back capacity that’s already been committed.
Continuous rightsizing, not one-time configuration. Resource requests set at deployment are almost always wrong six months later. Karpenter, OpenCost, and Kubecost are open-source options; Cast AI, ScaleOps, nOps, and PerfectScale automate the rightsizing itself. Cast AI reports its continuous rightsizing cuts provisioned CPU by roughly 50% on average across its customer base.
Regional spot placement, especially for T4-class inference. Cast AI’s survival-curve data shows T4 spot interruption risk ranging from about 10% over 24 hours in eu-west-3 to 80% in eu-central-1 and us-east-1. Region selection is a reliability decision, not just a latency one.
GPU sharing through MIG and time-slicing. Nvidia’s MIG feature partitions A100, H100, and H200 chips into isolated instances with dedicated compute and memory. vLLM and Dynamo implement continuous batching and disaggregated inference. Open primitives, no vendor contract required.
Disaggregated runtime. Ray lets CPU-bound data prep scale independently from GPU-bound training or inference.
Commitment rebalancing. Reserved Instances and Savings Plans drift as workloads change. Cast AI, nOps, and Vantage track utilization against committed capacity and adjust the split automatically.
The single most practical question most enterprises have not asked this year: do they actually need an H200 at all?
H200 is designed for very large models (70B+ parameters) with very long contexts (128k+ tokens), where its 141 GB of memory (nearly double the H100’s 80 GB) is what lets the chip handle the load without slowing down. For smaller models, fine-tuned derivatives, quantized inference, and most production AI that actually ships to customers, an H100 does the same job at roughly 40% less per GPU-hour, according to Cast AI. An A100 often works, too, at roughly 60% less. The era of a single general-purpose GPU as the default answer is ending. Chip selection is becoming a routing decision, workload by workload, rather than a generational procurement decision.
Gil’s own observation sharpens this. At 80% utilization, a B200 genuinely delivers better unit cost per token than an A100: more powerful per hour than it is more expensive per hour. At 5% utilization, the math inverts. The premium chip compounds the waste. Buying the newest chip while underusing it is the most expensive possible version of the FOMO loop.
The first lever is free, and it is a workload audit rather than a software purchase. No GPU needs to be released to run this lever. Every GPU-backed workload in production is worth reviewing against one question: is the chip it runs on actually matched to what it does. A surprising number of H200 purchases in 2026 will turn out to have been made because the allocation came through, not because the workload required it. Then fix runtime architecture before spending on more reserved capacity. Mix commodity and reserved tiers against the split instead of picking one.
Whether the broader GPU market eventually rebalances is a separate question, and not one worth betting a 2026 budget on. Supply could catch up. Memory capacity could ease. Specialized inference silicon could pull demand off the H200 tier. All of that is possible. None of it is certain. What is certain is that procurement and runtime are the same problem seen from two sides: FOMO drives over-commitment at the front end, and container architecture leaves the over-committed fleet idle at the back. Enterprises that treat them as one loop can break it. Enterprises that keep treating them as two separate budget items will keep paying to run their most expensive infrastructure at 5%.
Microsoft and OpenAI on Monday announced a sweeping overhaul of the partnership that has defined the commercial AI era, dismantling key pillars of exclusivity and revenue-sharing that bound the two companies together for years and replacing them with a looser, time-limited arrangement that gives both sides far more freedom to pursue rival relationships.
The amended agreement, disclosed simultaneously in blog posts from both companies, marks the most significant restructuring since Microsoft first invested $1 billion in OpenAI in 2019 — and it transforms what was once the most consequential exclusive technology alliance in a generation into something that more closely resembles a strategic but arm’s-length commercial relationship.
Under the new terms, Microsoft will no longer pay any revenue share to OpenAI when customers access OpenAI models through Azure. OpenAI, meanwhile, will continue paying a revenue share to Microsoft through 2030 — at the same 20 percent rate — but that obligation is now subject to a total cap. Microsoft retains a license to OpenAI’s intellectual property for models and products through 2032, but that license is now explicitly non-exclusive. And OpenAI, critically, can now serve all of its products to customers on any cloud provider — including Amazon Web Services and Google Cloud — ending the exclusivity that had been a cornerstone of the original deal.
“The rapid pace of innovation requires us to continue to evolve our partnership to benefit our customers and both companies,” Microsoft wrote in its blog post Monday. OpenAI echoed the framing, calling the amended agreement a move “grounded in flexibility, certainty, and a focus on delivering the benefits of AI broadly.”
The diplomatic language belies the drama that led to this moment — months of behind-the-scenes tension, competing deal announcements, public contradictions, and even the specter of litigation between two companies whose fates have been intertwined since the earliest days of the generative AI revolution.
To understand why Monday’s announcement matters so much, it helps to understand what came before it. When Microsoft poured its initial $1 billion into OpenAI in 2019, and then followed with a cumulative investment exceeding $13 billion, it secured something extraordinary: exclusive commercial access to OpenAI’s models and intellectual property. Azure became the sole cloud provider for OpenAI’s API products. Microsoft integrated OpenAI’s GPT models into everything from Bing to Office to GitHub Copilot. The arrangement was, by any measure, one of the most lopsided technology licensing deals in modern history — Microsoft got privileged access to the most capable AI models on the planet, and OpenAI got the capital and infrastructure it needed to scale.
The deal even contained an unusual provision: Microsoft’s exclusive rights would remain in force until OpenAI achieved artificial general intelligence, or AGI — a loosely defined milestone referring to AI systems that rival or exceed human intelligence across a broad range of tasks. OpenAI’s board retained the authority to declare when AGI had been reached, at which point certain commercial terms would change. It was, in effect, a philosophical tripwire embedded in a business contract.
That structure worked well enough when OpenAI was a research lab with a modest commercial footprint. But as ChatGPT exploded into the mainstream in late 2022 and OpenAI’s annualized revenue rocketed into the billions, the constraints began to chafe. OpenAI found itself locked into a single cloud ecosystem at precisely the moment when enterprises — its fastest-growing customer segment — were demanding multi-cloud flexibility. In an internal memo earlier this month, OpenAI’s revenue chief Denise Dresser put it bluntly, telling staff that the Microsoft partnership had “limited our ability to meet enterprises where they are,” according to a report from The Verge.
The proximate cause of Monday’s restructuring was not a philosophical disagreement about AI safety or corporate governance. It was a $50 billion check from Amazon. In February, OpenAI announced that Amazon would invest up to $50 billion in the company — $15 billion upfront, with another $35 billion to follow when certain unspecified conditions were met. In exchange, OpenAI agreed to expand its existing cloud agreement with AWS by $100 billion over eight years and, most controversially, committed to making AWS the exclusive third-party distribution provider for Frontier, its new enterprise agent-building platform. OpenAI also agreed to co-develop “stateful runtime technology” on AWS Bedrock, the infrastructure layer that allows AI agents to maintain memory and context over extended tasks.
The problem was that OpenAI’s existing contract with Microsoft almost certainly prohibited these arrangements. Microsoft held exclusive rights to any OpenAI product accessed through an API — a category that plainly included Frontier. On the very day OpenAI announced the Amazon deal, Microsoft issued a pointed public statement insisting that “Azure remains the exclusive cloud provider of stateless OpenAI APIs” and that “OpenAI’s first party products, including Frontier, will continue to be hosted on Azure.” The contradiction between the two announcements was stark, and it created immediate legal exposure. The Financial Times reported in March that Microsoft was actively considering legal action to enforce its contractual rights. The situation placed OpenAI in an impossible position: it had made promises to Amazon that it seemingly could not keep under the terms of its Microsoft agreement.
Monday’s deal resolves that impasse entirely. By converting Microsoft’s license from exclusive to non-exclusive and explicitly granting OpenAI the right to serve products on any cloud, the new terms retroactively validate the Amazon arrangement and eliminate the legal overhang. Amazon CEO Andy Jassy wasted no time celebrating. “We’re excited to make OpenAI’s models available directly to customers on Bedrock in the coming weeks, alongside the upcoming Stateful Runtime Environment,” he wrote on X, adding that the company would share more details at an event in San Francisco on Tuesday.
The financial mechanics of the new deal deserve careful parsing, because they reveal which side gave up what — and who came out ahead. Under the old arrangement, money flowed in both directions. When customers bought ChatGPT subscriptions or accessed OpenAI models through their own applications, OpenAI paid Microsoft a cut — reportedly 20 percent. Conversely, when enterprise customers accessed OpenAI models through Azure’s API, Microsoft paid OpenAI a share of that revenue. This bilateral structure reflected the deep integration between the two companies: Microsoft was simultaneously OpenAI’s investor, cloud provider, distribution partner, and largest customer.
The new deal makes the cash flow one-directional. Microsoft stops paying OpenAI entirely. OpenAI continues paying Microsoft its 20 percent share, but only through 2030, and now subject to a total cap whose precise dollar figure has not been disclosed. Given that OpenAI’s revenue is growing rapidly — the company was reportedly on pace to generate tens of billions annually — that cap could become material relatively quickly.
For Microsoft, the trade-off is straightforward: it sacrifices the exclusivity that made Azure the only gateway to OpenAI’s models, but it gains immediate financial relief by eliminating its outbound revenue-share payments while continuing to collect inbound payments for several more years. And it retains approximately 27 percent ownership of OpenAI’s for-profit entity, meaning it participates in the company’s growth regardless of which cloud serves the workloads. Last quarter alone, Microsoft reported $7.5 billion in revenue from its OpenAI investment in a single quarter, according to TechCrunch’s reporting. For OpenAI, the calculus is different. It accepts a continued obligation to pay Microsoft through 2030, but it gains the commercial freedom to sell everywhere — a freedom that is arguably worth far more than the revenue-share savings. Enterprise customers overwhelmingly operate in multi-cloud environments. Being locked into Azure was not just a technical constraint; it was a sales objection that OpenAI’s competitors, particularly Anthropic and Google, exploited relentlessly.
One of the more philosophically intriguing aspects of Monday’s announcement is what it does to the AGI provision that once governed the partnership. Under the original agreement, Microsoft’s exclusive commercial rights were tied to a trigger: if OpenAI’s board determined that the company had achieved AGI, certain terms — including Microsoft’s access to the most advanced models — would change. The provision was meant to ensure that a truly superintelligent system would remain under the nonprofit board’s control rather than being commercially exploited. In practice, it created perverse incentives: OpenAI had a financial reason to never declare AGI, and Microsoft had a financial reason to argue that AGI had not been reached regardless of what the technology could actually do.
The new deal sidesteps this entirely. Microsoft’s license now runs through a fixed calendar date — 2032 — “independent of OpenAI’s technology progress,” as the companies put it. The AGI trigger, a concept that once sat at the philosophical heart of the partnership, has been replaced by a spreadsheet. Andrew Curran, a close observer of OpenAI’s governance, noted on X that language defining AGI had been removed from OpenAI’s website, sharing a screenshot showing the change. The move drew sharp reactions. One commenter observed that “removing the definition = removing the accountability. whoever controls when AGI is declared controls a lot of commercial terms.”
The shift reflects a broader maturation — or perhaps disillusionment — within the AI industry regarding AGI as a meaningful commercial or governance concept. When the original deal was struck, AGI felt like a distant, almost mythical threshold. Now, with models like GPT-5.5 demonstrating increasingly general capabilities, the term has become more of a marketing slogan than a technical benchmark. Replacing it with fixed dates and dollar caps is, in some sense, an admission that the industry has moved beyond the framework that once defined this partnership.
The most immediate beneficiary of the new arrangement is the enterprise customer. For years, organizations that wanted access to OpenAI’s models had essentially one option: Azure. That constraint is now gone. Within weeks, according to Jassy, OpenAI’s models will be available on AWS Bedrock alongside the stateful runtime environment that powers long-running AI agents. Google Cloud is presumably not far behind.
This multi-cloud availability arrives at a moment when the AI infrastructure market is undergoing rapid consolidation and expansion simultaneously. Meta recently committed $48 billion to cloud providers CoreWeave and Nebius. Amazon’s investment in OpenAI, combined with its existing relationship with Anthropic — in which Amazon has invested up to $4 billion — positions AWS as a model-agnostic platform where enterprises can mix and match AI capabilities. Microsoft, meanwhile, has developed its own relationship with Anthropic, using Claude to power agentic products — a hedge against the very OpenAI dependency it spent billions creating.
The competitive dynamics are now genuinely complex. Microsoft competes with OpenAI in AI products (Copilot vs. ChatGPT), partners with OpenAI’s rival Anthropic, and remains OpenAI’s largest shareholder. OpenAI sells on Azure, AWS, and soon everywhere else, while building its own data centers. Amazon invests in both OpenAI and Anthropic. Google builds its own models while also hosting competitors on Vertex AI. Jehangeer Hasan, a technology commentator, captured the mood on X, calling the announcement a “notable shift in the cloud AI landscape” that signals “intensifying multi-cloud competition and a push toward giving developers more flexibility instead of locking them into a single ecosystem.” Chris Alexander, an engineer, offered a more candid assessment: “honestly Azure’s OpenAI endpoints are so unreliable, we mostly just hit you all directly,” adding that “it would be nice to have options in AWS or GCP for sure.”
Several open questions remain. The precise dollar amount of the revenue-share cap has not been disclosed, and it will matter enormously as OpenAI’s revenue scales. The meaning of “first on Azure” — whether it implies a meaningful exclusivity window or merely simultaneous availability — remains deliberately ambiguous. And OpenAI’s own infrastructure ambitions, including plans to build proprietary data centers, could eventually reduce its dependence on any third-party cloud, including Azure.
Microsoft’s position, while less dominant than before, is not as diminished as some early commentary suggested. It remains OpenAI’s primary cloud provider, its largest shareholder, and a licensee of its technology through the end of the decade. It has diversified its own AI strategy with investments in Anthropic, its own Phi and MAI model families, and deep integration of AI across its product portfolio. The company reported $7.5 billion in OpenAI-related revenue last quarter — a figure that demonstrates the sheer financial scale of the relationship even in its loosened form.
For OpenAI, the new agreement is a coming-of-age moment. The company that once depended on Microsoft for everything — capital, compute, distribution, and credibility — now operates as an independent force capable of striking multi-billion-dollar deals with Microsoft’s biggest rivals. Sam Altman announced the changes on X with characteristic brevity: “We have updated our partnership with Microsoft.”
Seven years ago, when Microsoft CEO Satya Nadella and Altman first shook hands on a deal to commercialize artificial intelligence, the arrangement rested on the assumption that OpenAI needed Microsoft more than Microsoft needed OpenAI. Every clause — the exclusivity, the AGI trigger, the revenue share — reflected that original imbalance. Monday’s restructuring is proof that the assumption no longer holds. The partnership that launched the generative AI revolution has survived, but the power dynamics that created it have not. In the AI industry, it turns out, the only thing that moves faster than the technology is the leverage.
The most expensive AI failure I have seen in enterprise deployments did not produce an error. No alert fired. No dashboard turned red. The system was fully operational, it was just consistently, confidently wrong. That is the reliability gap. And it is the problem most enterprise AI programs are not built to catch.
We have spent the last two years getting very good at evaluating models: benchmarks, accuracy scores, red-team exercises, retrieval quality tests. But in production, the model is rarely where the system breaks. It breaks in the infrastructure layer, the data pipelines feeding it, the orchestration logic wrapping it, the retrieval systems grounding it, the downstream workflows trusting its output. That layer is still being monitored with tools designed for a different kind of software.
Here’s what makes this problem hard to see: Operationally healthy and behaviorally reliable are not the same thing, and most monitoring stacks cannot tell the difference.
A system can show green across every infrastructure metric, latency within SLA, throughput normal, error rate flat, while simultaneously reasoning over retrieval results that are six months stale, silently falling back to cached context after a tool call degrades, or propagating a misinterpretation through five steps of an agentic workflow. None of that shows up in Prometheus. None of it trips a Datadog alert.
The reason is straightforward: Traditional observability was built to answer the question “is the service up?” Enterprise AI requires answering a harder question: “Is the service behaving correctly?” Those are different instruments.
|
What teams typically measure |
What actually drives AI infrastructure failure |
|
Uptime / latency / error rate |
Retrieval freshness and grounding confidence |
|
Token usage |
Context integrity across multi-step workflows |
|
Throughput |
Semantic drift under real-world load |
|
Model benchmark scores |
Behavioral consistency when conditions degrade |
|
Infrastructure error rate |
Silent partial failure at the reasoning layer |
Closing this gap requires adding a behavioral telemetry layer alongside the infrastructure one — not replacing what exists, but extending it to capture what the model actually did with the context it received, not just whether the service responded.
Across enterprise AI deployments in network operations, logistics, and observability platforms, I see four failure patterns repeat with enough consistency to name them.
The first is context degradation. The model reasons over incomplete or stale data in a way that is invisible to the end user. The answer looks polished. The grounding is gone. Detection usually happens weeks later, through downstream consequences rather than system alerts.
The second is orchestration drift. Agentic pipelines rarely fail because one component breaks. They fail because the sequence of interactions between retrieval, inference, tool use, and downstream action starts to diverge under real-world load. A system that looked stable in testing behaves very differently when latency compounds across steps and edge cases stack.
The third is a silent partial failure. One component underperforms without crossing an alert threshold. The system degrades behaviorally before it degrades operationally. These failures accumulate quietly and surface first as user mistrust, not incident tickets. By the time the signal reaches a postmortem, the erosion has been happening for weeks.
The fourth is the automation blast radius. In traditional software, a localized defect stays local. In AI-driven workflows, one misinterpretation early in the chain can propagate across steps, systems, and business decisions. The cost is not just technical. It becomes organizational, and it is very hard to reverse.
Metrics tell you what happened. They rarely tell you what almost happened.
Traditional chaos engineering asks the right kind of question: What happens when things break? Kill a node. Drop a partition. Spike CPU. Observe. Those tests are necessary, and enterprises should run them.
But for AI systems, the most dangerous failures are not caused by hard infrastructure faults. They emerge at the interaction layer between data quality, context assembly, model reasoning, orchestration logic, and downstream action. You can stress the infrastructure all day and never surface the failure mode that costs you the most.
What AI reliability testing needs is an intent-based layer: Define what the system must do under degraded conditions, not just what it should do when everything works. Then test the specific conditions that challenge that intent. What happens if the retrieval layer returns content that is technically valid but six months outdated? What happens if a summarization agent loses 30% of its context window to unexpected token inflation upstream? What happens if a tool call succeeds syntactically but returns semantically incomplete data? What happens if an agent retries through a degraded workflow and compounds its own error with each step?
These scenarios are not edge cases. They are what production looks like. This is the framework I have applied in building reliability systems for enterprise infrastructure: Intent-based chaos level creation for distributed computing environments. The key insight: Intent defines the test, not just the fault.
None of this requires reinventing the stack. It requires extending four things.
Add behavioral telemetry alongside infrastructure telemetry. Track whether responses were grounded, whether fallback behavior was triggered, whether confidence dropped below a meaningful threshold, whether the output was appropriate for the downstream context it entered. This is the observability layer that makes everything else interpretable.
Introduce semantic fault injection into pre-production environments. Deliberately simulate stale retrieval, incomplete context assembly, tool-call degradation, and token-boundary pressure. The goal is not theatrical chaos. The goal is finding out how the system behaves when conditions are slightly worse than your staging environment — which is always what production is.
Define safe halt conditions before deployment, not after the first incident. AI systems need the equivalent of circuit breakers at the reasoning layer. If a system cannot maintain grounding, validate context integrity, or complete a workflow with enough confidence to be trusted, it should stop cleanly, label the failure, and hand control to a human or a deterministic fallback. A graceful halt is almost always safer than a fluent error. Too many systems are designed to keep going because confident output creates the illusion of correctness.
Assign shared ownership for end-to-end reliability. The most common organizational failure is a clean separation between model teams, platform teams, data teams, and application teams. When the system is operationally up but behaviorally wrong, no one owns it clearly. Semantic failure needs an owner. Without one, it accumulates.
For the last two years, the enterprise AI differentiator has been adoption — who gets to production fastest. That phase is ending. As models commoditize and baseline capability converges, competitive advantage will come from something harder to copy: The ability to operate AI reliably at scale, in real conditions, with real consequences.
Yesterday’s differentiator was model adoption. Today’s is system integration. Tomorrow’s will be reliability under production stress.
The enterprises that get there first will not have the most advanced models. They will have the most disciplined infrastructure around them — infrastructure that was tested against the conditions it would actually face, not the conditions that made the pilot look good.
The model is not the whole risk. The untested system around it is.
Sayali Patil is an AI infrastructure and product leader.
Traditional software is predictable: Input A plus function B always equals output C. This determinism allows engineers to develop robust tests. On the other hand, generative AI is stochastic and unpredictable. The exact same prompt often yields different results on Monday versus Tuesday, breaking the traditional unit testing that engineers know and love.
To ship enterprise-ready AI, engineers cannot rely on mere “vibe checks” that pass today but fail when customers use the product. Product builders need to adopt a new infrastructure layer: The AI Evaluation Stack.
This framework is informed by my extensive experience shipping AI products for Fortune 500 enterprise customers in high-stakes industries, where “hallucination” is not funny — it’s a huge compliance risk.
Traditional software tests are binary assertions (pass/fail). While some AI evals use binary asserts, many evaluate on a gradient. An eval is not a single script; it is a structured pipeline of assertions — ranging from strict code syntax to nuanced semantic checks — that verify the AI system’s intended function.
To build a robust, cost-effective pipeline, asserts must be separated into two distinct architectural layers:
A surprisingly large share of production AI failures aren’t semantic “hallucinations” — they are basic syntax and routing failures. Deterministic assertions serve as the pipeline’s first gate, using traditional code and regex to validate structural integrity.
Instead of asking if a response is “helpful,” these assertions ask strict, binary questions:
Did the model generate the correct JSON key/value schema?
Did it invoke the correct tool call with the required arguments?
Did it successfully slot-fill a valid GUID or email address?
// Example: Layer 1 Deterministic Tool Call Assertion
{
“test_scenario”: “User asks to look up an account”,
“assertion_type”: “schema_validation”,
“expected_action”: “Call API: get_customer_record”,
“actual_ai_output”: “I found the customer.”,
“eval_result”: “FAIL – AI hallucinated conversational text instead of generating the required API payload.”
}
In the example above, the test failed instantly because the model generated conversational text instead of the required tool call payload.
Architecturally, deterministic assertions must be the first layer of the stack, operating on a computationally inexpensive “fail-fast” principle. If a downstream API requires a specific schema, a malformed JSON string is a fatal error. By failing the evaluation immediately at this layer, engineering teams prevent the pipeline from triggering expensive semantic checks (Layer 2) or wasting valuable human review time (Layer 3).
When deterministic assertions pass, the pipeline must evaluate semantic quality. Because natural language is fluid, traditional code cannot easily assert if a response is “helpful” or “empathetic.” This introduces model-based evaluation, commonly referred to as “LLM-as-a-Judge” or “LLM-Judge.”
While using one non-deterministic system to evaluate another seems counterintuitive, it is an exceptionally powerful architectural pattern for use cases requiring nuance. It is virtually impossible to write a reliable regex to verify if a response is “actionable” or “polite.” While human reviewers excel at this nuance, they cannot scale to evaluate tens of thousands of CI/CD test cases. Thus, the LLM-as-a-Judge becomes the scalable proxy for human discernment.
However, model-based assertions only yield reliable data when the LLM-as-a-Judge is provisioned with three critical inputs:
A state-of-the-art reasoning model: The Judge must possess superior reasoning capabilities compared to the production model. If your app runs on a smaller, faster model for latency, the judge must be a frontier reasoning model to approximate human-level discernment.
A strict assessment rubric: Vague evaluation prompts (“Rate how good this answer is”) yield noisy, stochastic evaluations. A robust rubric explicitly defines the gradients of failure and success. (For example, a “Helpfulness” rubric should define Score 1 as an irrelevant refusal, Score 2 as addressing the prompt but lacking actionable steps, and Score 3 as providing actionable next steps strictly within context.)
Ground truth (golden outputs): While the rubric provides the rules, a human-vetted “expected answer” acts as the answer key. When the LLM-Judge can compare the production model’s output against a verified Golden Output, its scoring reliability increases dramatically.
A robust evaluation architecture requires two complementary pipelines. The online pipeline monitors post-deployment telemetry, while the offline pipeline provides the foundational baseline and deterministic constraints required to evaluate stochastic models safely.
The offline pipeline’s primary objective is regression testing — identifying failures, drift, and latency before production. Deploying an enterprise LLM feature without a gating offline evaluation suite is an architectural anti-pattern; it is the equivalent of merging uncompiled code into a main branch.
The offline lifecycle begins by curating a “golden dataset” — a static, version-controlled repository of 200 to 500 test cases representing the AI’s full operational envelope. Each case pairs an exact input payload with an expected “golden output” (ground truth).
Crucially, this dataset must reflect expected real-world traffic distributions. While most cases cover standard “happy-path” interactions, engineers must systematically incorporate edge cases, jailbreaks, and adversarial inputs. Evaluating “refusal capabilities” under stress remains a strict compliance requirement.
Example test case payload (standard tool use):
Input: “Schedule a 30-minute follow-up meeting with the client for next Tuesday at 10 a.m.”
Expected output (golden): The system successfully invokes the schedule_meeting tool with the correct JSON payload: {“duration_minutes”: 30, “day”: “Tuesday”, “time”: “10 AM”, “attendee”: “client_email”}.
While manually curating hundreds of edge cases is tedious, the process can be accelerated with synthetic data generation pipelines that use a specialized LLM to produce diverse TSV/CSV test payloads. However, relying entirely on AI-generated test cases introduces the risk of data contamination and bias. A human-in-the-loop (HITL) architecture is mandatory at this stage; domain experts must manually review, edit, and validate the synthetic dataset to ensure it accurately reflects real-world user intent and enterprise policy before it is committed to the repository.
Once the dataset is curated, engineers must design the evaluation criteria to compute a composite score for each model output. A robust architecture achieves this by assigning weighted points across a hybrid of Layer 1 (deterministic) and Layer 2 (model-based) asserts.
Consider an AI agent executing a “send email” tool. An evaluation framework might utilize a 10-point scoring system:
Layer 1: Deterministic asserts (6 points): Did the agent invoke the correct tool? (2 pts). Did it produce a valid JSON object? (2 pts). Does the JSON strictly adhere to the expected schema? (2 pts).
Layer 2: Model-based asserts (4 points): (Note: Semantic rubrics must be highly use-case specific). Does the subject line reflect user intent? (1 pt). Does the email body match expected outputs without hallucination? (1 pt). Were CC/BCC fields leveraged accurately? (1 pt). Was the appropriate priority flag inferred? (1 pt).
To understand why the LLM-Judge awarded these points, the engineer must prompt the judge to supply its reasoning for each score. This is crucial for debugging failures.
The passing threshold and short-circuit logic
In this example, an 8/10 passing threshold requires 8 points for success. Crucially, the evaluation pipeline must enforce strict short-circuit evaluation (fail-fast logic). If the model fails any deterministic assertion — such as generating a malformed JSON schema — the system must instantly fail the entire test case (0/10). There is zero architectural value in invoking an expensive LLM-Judge to assess the semantic “politeness” of an email if the underlying API call is structurally broken.
Using an evaluation infrastructure of choice, the system executes the offline pipeline — typically integrated as a blocking CI/CD step during a pull request. The infrastructure iterates through the golden dataset, injecting each test payload into the production model, capturing the output, and executing defined assertions against it.
Each output is scored against the passing threshold. Once batch execution is complete, results are aggregated into an overall pass rate. For enterprise-grade applications, the baseline pass rate must typically exceed 95%, scaling to 99%-plus for strict compliance or high-risk domains.
Based on aggregated failure data, engineering teams conduct a root-cause analysis of failing test cases. This assessment drives iterative updates to core components: refining system prompts, modifying tool descriptions, augmenting knowledge sources, or adjusting hyperparameters (like temperature or top-p). Continuous optimization remains best practice even after achieving a 95% pass rate.
Crucially, any system modification necessitates a full regression test. Because LLMs are inherently non-deterministic, an update intended to fix one specific edge case can easily cause unforeseen degradations in other areas. The entire offline pipeline must be rerun to validate that the update improved quality without introducing regressions.
While the offline pipeline acts as a strict pre-deployment gatekeeper, the online pipeline is the post-deployment telemetry system. Its objective is to monitor real-world behavior, capturing emergent edge cases, and quantifying model drift. Architects must instrument applications to capture five distinct categories of telemetry:
Direct, deterministic feedback indicating model performance:
Thumbs up/down: Disproportionate negative feedback is the most immediate leading indicator of system degradation, directing immediate engineering investigation.
Verbatim in-app feedback: Systematically parsing written comments identifies novel failure modes to integrate back into the offline “golden dataset.”
Behavioral telemetry reveals silent failures where users give up without explicit feedback:
Regeneration and retry rates: High frequencies of retries indicate the initial output failed to resolve user intent.
Apology rate: Programmatically scanning for heuristic triggers (“I’m sorry”) detects degraded capabilities or broken tool routing.
Refusal rate: Artificially high refusal rates (“I can’t do that”) indicate over-calibrated safety filters rejecting benign user queries.
Because deterministic code checks execute in milliseconds, teams can seamlessly reuse Layer 1 offline asserts (schema conformity, tool validity) to synchronously evaluate 100% of production traffic. Logging these pass/fail rates instantly detects anomalous spikes in malformed outputs — the earliest warning sign of silent model drift or provider-side API changes.
If strict data privacy agreements (DPAs) permit logging user inputs, teams can deploy model-based asserts. Architecturally, production LLM-Judges must never execute synchronously on the critical path, which doubles latency and compute costs. Instead, a background LLM-Judge asynchronously samples a fraction (5%) of daily sessions, grading outputs against the offline rubric to generate a continuous quality dashboard.
Evaluation pipelines are not “set-it-and-forget-it” infrastructure. Without continuous updates, static datasets suffer from “rot” (concept drift) as user behavior evolves and customers discover novel use cases.
For example, an HR chatbot might boast a pristine 99% offline pass rate for standard payroll questions. However, if the company suddenly announces a new equity plan, users will immediately begin prompting the AI about vesting schedules — a domain entirely missing from the offline evaluations.
To make the system smarter over time, engineers must architect a closed feedback loop that mines production telemetry for continuous improvement.
The continuous improvement workflow:
Capture: A user triggers an explicit negative signal (a “thumbs down”) or an implicit behavioral flag in production.
Triage: The specific session log is automatically flagged and routed for human review.
Root-cause analysis: A domain expert investigates the failure, identifies the gap, and updates the AI system to successfully handle similar requests.
Dataset augmentation: The novel user input, paired with the newly corrected expected output, is appended to the offline Golden Dataset alongside several synthetic variations.
Regression testing: The model is continuously re-evaluated against this newly discovered edge case in all future runs.
Building an evaluation pipeline without monitoring production logs and updating datasets is fundamentally insufficient. Users are unpredictable. Evaluating on stale data creates a dangerous illusion: High offline pass rates masking a rapidly degrading real-world experience.
In the era of generative AI, a feature or product is no longer “done” simply because the code compiles and the prompt returns a coherent response. It is only done when a rigorous, automated evaluation pipeline is deployed and stable — and when the model consistently passes against both a curated golden dataset and newly discovered production edge cases.
This guide has equipped you with a comprehensive blueprint for building that reality. From architecting offline regression pipelines and online telemetry to the continuous feedback flywheel and navigating enterprise anti-patterns, you now have the structural foundation required to deploy AI systems with greater confidence.
Now, it is your turn. Share this framework with your engineering, product, and legal teams to establish a unified, cross-functional standard for AI quality in your organization. Stop guessing whether your models are degrading in production, and start measuring.
Derah Onuorah is a Microsoft senior product manager.
Cirrascale Cloud Services today announced it has expanded its partnership with Google Cloud to deliver the Gemini model on-premises through Google Distributed Cloud, making it the first neocloud provider to offer Google’s most advanced AI model as a fully private, disconnected appliance. The announcement, timed to coincide with Google Cloud Next 2026 in Las Vegas, addresses a stubborn problem that has plagued regulated industries since the generative AI boom began: how to access frontier-class AI models without surrendering control of your data.
The offering packages Gemini into a Dell-manufactured, Google-certified hardware appliance equipped with eight Nvidia GPUs and wrapped in confidential computing protections. Enterprises and government agencies can deploy the system inside Cirrascale’s data centers or their own facilities, fully disconnected from the internet and from Google’s cloud infrastructure. The product enters preview immediately, with general availability expected in June or July.
In an exclusive interview with VentureBeat ahead of the announcement, Dave Driggers, CEO of Cirrascale Cloud Services, described the deployment as “the next step of the partnership” and “being able to offer their most important model they have, which is Gemini.” He was emphatic about what customers would be getting: “It is full blown Gemini. It’s not pulled,” he told VentureBeat. “Nothing’s missing from it, and it’ll be available in a private scenario, so that we can guarantee them that their data is secure, their inputs are secure, their outputs are secure.”
The move signals a deepening shift in the enterprise AI market, where the most capable models are migrating out of hyperscaler data centers and into customers’ own racks — a reversal of the cloud computing orthodoxy that defined the past decade.
For years, organizations in financial services, healthcare, defense and government faced a binary choice: access the most powerful AI models through public cloud APIs, exposing sensitive data to third-party infrastructure, or settle for less capable open-source models they could host themselves. Cirrascale’s new offering attempts to eliminate that tradeoff entirely.
Driggers described how the trust problem escalated in stages. First, companies worried about handing their proprietary data to hyperscalers. Then came a deeper realization. “They started realizing, holy crap, when my users type stuff in, they’re giving private information away — and the output is private too,” Driggers told VentureBeat. “And then the hyperscalers said, ‘Your prompts and the responses? That’s our stuff. We need that in order to answer your question.'” That was the moment, he argued, when the demand for fully private AI became impossible to ignore.
Unlike Google Distributed Cloud, which Google already offers as its own on-premises cloud extension, the Cirrascale deployment places the actual model — weights and all — outside of Google’s infrastructure entirely. “Google doesn’t own this hardware. We own the hardware, or the customer owns the hardware,” Driggers said. “It is completely outside of Google.”
Driggers drew a sharp distinction between this offering and what competitors provide. When asked about Microsoft Azure’s on-premises deployments with OpenAI models and AWS Outposts, he was blunt: “Those are a lot different. This is the actual model being deployed on prem outside of their cloud. It’s not a cut down version. It’s the actual model.”
The technical underpinnings of the deployment reveal how seriously both Google and Cirrascale are treating the security question. The Gemini model resides entirely in volatile memory — not on persistent storage. “As soon as the power is off, the model is gone,” Driggers explained. User sessions operate through caches that clear automatically when a session ends. “A company’s user inputs, once that session’s over, they’re gone. They can be saved, but by default, they’re gone,” he said.
Perhaps the most striking security feature is what happens when someone attempts to tamper with the appliance. Driggers described a mechanism that effectively renders the machine inoperable: “You do anything that is against confidential compute, and it’s gone. Not only does the machine turn off, and therefore the model is gone, it actually puts in a marker that says, ‘You violated the confidential compute.’ That machine has to come back to us, or back to Dell or back to Google.” He characterized the appliance as something that “does time bomb itself if something goes wrong.”
This level of protection reflects Google’s own anxiety about releasing its flagship model’s weights into environments it doesn’t control. The appliance is effectively a vault: the model runs inside it, but nobody — not even the customer — can extract or inspect the weights. The confidential computing envelope ensures that even physical possession of the hardware doesn’t grant access to the model’s intellectual property.
When Google releases a new version of Gemini, the appliance needs to reconnect — but only briefly, and through a private channel. “It does have to get connected back to Google to load the new model. But that can go via a private connection,” Driggers said. For the most security-sensitive customers who can never allow their machine to connect to an outside network, Cirrascale offers a physical swap: “The server will be unplugged, purged, all the data gone, guaranteed it’s gone, a new server will show up with a new version of the model.”
Driggers identified three primary drivers of demand: trust, security and guaranteed performance. Financial services institutions top the list. “They’ve got regulatory issues where they can’t have something out of their control. They’ve got to be the one who determines where everything is. It’s got to be air gap,” Driggers said. The minimum deployment footprint — a single eight-GPU server — makes the product accessible in a way that Google’s own private offerings do not. Running Gemini on Google’s TPU-based infrastructure, Driggers noted, requires a much larger commitment. “If you want a private [instance] from Google, they require a much bigger bite, because to build something private for you, Google requires a gigantic footprint. Here we can do it down to a single machine.”
Beyond finance, Driggers pointed to drug discovery, medical data, public-sector research, and any business handling personal information. He also flagged an increasingly critical use case: data sovereignty. “How about your business that’s doing business outside of the United States, and now you’ve got data sovereignty laws in places where GCP is not? We can provide private Gemini in these smaller countries where the data can’t leave.”
The public sector is another major target. Cirrascale launched a dedicated Government Services division in March as part of its earlier partnership with Google Public Sector around the GPAR (Google Public Sector Program for Accelerated Research) initiative. That program provides higher education and research institutions access to AI tools including AlphaFold, AI Co-Scientist, and Gemini Enterprise for Education. Today’s announcement extends that relationship from the research tooling layer to the model itself.
The performance guarantee is the third pillar. Driggers noted that frontier models accessed through public APIs deliver inconsistent response times — a problem for mission-critical business applications. The private deployment eliminates that variability. Cirrascale layers management software on top of the Gemini appliance that allows administrators to prioritize users, allocate tokens by role, adjust context window sizes, and load-balance across multiple appliances and regions. “Your primary data scientists or your programmers may need to have really large context windows and get priority, especially maybe nine to five,” Driggers explained, “but yet, the rest of the time, they want to share the Gemini experience over a wider group of people.” He also noted that agentic AI workloads, which can run around the clock, benefit from the ability to consume unused capacity during off-peak hours — a scheduling flexibility that public cloud deployments don’t easily support.
The pricing model reflects Cirrascale’s broader philosophy of meeting customers where they are. Driggers described several consumption options: seat-based licensing (with both enterprise and standard tiers), per-token billing, and flat “all-you-can-eat” pricing per appliance. The minimum commitment is a single dedicated server — the appliances are not shared between customers in any configuration. “We’ll meet the customer, what they’re used to,” Driggers said. “If they’re currently taking a seat license, we’ll create a seat license for them.”
Customers can also choose to purchase the hardware outright while still consuming Gemini as a managed service, an arrangement Cirrascale has offered since its earliest days in the AI wave. Driggers said OpenAI has been a customer since 2016 or 2017, and in that engagement, OpenAI purchased its own GPUs while Cirrascale “took those GPUs, incorporated them into our servers and storage and networking, and then presented it back as a cloud service to them so they didn’t have to manage anything.”
That flexible ownership model is particularly relevant for universities and government-funded research institutions, where mandates often require a specific mix of capital expenditure, operating expenditure, and personnel investment. “A lot of government funding requires a mixture of CapEx, OPEX and employment development,” Driggers said. “So we allow that as well.”
Cirrascale’s announcement arrives during a period of explosive growth for the neocloud sector — the tier of specialized AI cloud providers that sit between the hyperscalers and traditional hosting companies. The neocloud market is projected to be worth $35.22 billion in 2026 and is growing at a compound annual growth rate of 46.37%, according to Mordor Intelligence. Leading neocloud providers include CoreWeave, Crusoe Cloud, Lambda, Nebius and Vultr, and these companies specialize in GPU-as-a-Service for AI and high-performance computing workloads.
But Cirrascale occupies a different niche within this booming category. While companies like CoreWeave have focused primarily on providing raw GPU compute at scale — CoreWeave boasts a $55.6 billion backlog — Cirrascale has positioned itself around private AI, managed services and longer-term engagements rather than on-demand elastic compute. Driggers described the company as “not an on-demand place” but rather a provider focused on “longer-term workloads where we’re really competing against somebody doing it back on prem.”
The company’s history supports that claim. Cirrascale traces its roots to a hardware company that “designed the world’s first eight GPU server in 2012 before anybody thought you’d ever need eight GPUs in a box,” as Driggers put it. It pivoted to pure cloud services roughly eight years ago and has since built a client roster that includes the Allen Institute for AI, which in August 2025 tapped Cirrascale as the managed services provider for a $152 million open AI initiative funded by the National Science Foundation and Nvidia. Earlier this month, Cirrascale announced a three-way alliance with Rafay Systems and Cisco to deliver end-to-end enterprise AI solutions combining Cirrascale’s inference platform, Rafay’s GPU orchestration, and Cisco’s networking and compute hardware.
The Gemini partnership is the highest-profile move yet — and it taps into a broader industry current. The push to move frontier AI out of the public cloud and into private infrastructure is no longer a niche demand. Industry analysts predict that by 2027, 40% of AI model training and inference will occur outside public cloud environments. That projection helps explain why Google is willing to let its crown-jewel model run on hardware it doesn’t own, in data centers it doesn’t operate, managed by a company in San Diego. The alternative — watching regulated enterprises default to open-source models or to Microsoft’s Azure OpenAI Service — is apparently a worse outcome.
The announcement also carries major implications for Google’s competitive positioning. Microsoft has built its enterprise AI strategy around the Azure OpenAI Service and its deep partnership with OpenAI, while AWS has invested in Amazon Bedrock and its own on-premises solutions through Outposts. Google Cloud Platform still trails both rivals in market share, though Q4 cloud revenue rose 48% year-over-year. Enabling Gemini to run on third-party infrastructure via partners like Cirrascale broadens its distribution surface in exactly the segments — government, finance, healthcare — where Microsoft and Amazon have historically held advantages. For Cirrascale, the partnership represents a chance to differentiate sharply in a market where most neoclouds are competing on GPU availability and price.
Driggers expects rapid uptake in the second half of 2026. “It’s going to be crazy towards the end of this year,” he said. “Major banks will finally do stuff like this, because they can secure it. They can do it globally. Big research institutions who have labs all over the world will do these types of things.” He predicted other frontier model providers will follow with similar offerings soon, and he doesn’t see Gemini as the end of the story. “We really think that the enterprise have been waiting for private AI, not just Gemini, but all sorts of private AI,” Driggers said.
That may be the most telling line of all. For three years, the AI revolution has been defined by a simple bargain: send your data to the cloud and get intelligence back. Cirrascale’s bet — and increasingly, Google’s — is that the biggest customers in the world are done accepting those terms. The most powerful AI on the planet is now available on a single locked box that can sit in a bank vault, a university basement, or a government facility in a country where Google has no data center. The cloud, it turns out, is finally ready to come back down to earth.

Examining how a U.S. Interregional Transmission Overlay could address aging grid infrastructure, surging demand, and renewable integration challenges.What Attendees will LearnWhy the current regional grid structure is approaching its limits — Explore …