At Kilo Code, engineers are reading or writing code themselves only about 1% of the time now, according to co-founder Emilie Schario — the rest is agents. That shift is forcing new questions onto dev teams: which systems are safe to hand over, who cleans up when models goof up, how to support multi-model architectures, and whether skyrocketing token bills mean real progress or just burned IT budget.
As far as tech leads from Replit, Kilo Code, and Symbotic are concerned, it’s a natural — and welcome — evolution as agentic AI becomes embedded into more and more enterprise workflows.
“Unless something’s really broken or debugging, 99% of the time engineers are not reading or writing code anymore,” Emilie Schario, co-founder of Kilo Code, said at VB Transform 2026.
For Jared Go, distinguished engineer for AI and cloud at warehouse automation company Symbotic, the current moment is about directing the focus of AI. “These are my criteria,” he said. “Let’s look at it from the lens of security, elegance, clean, concise code, water tightness.” That way, AI does most of the heavy lifting, and human code review isn’t as critical.
Human involvement becomes necessary further down the line, Go noted, because agents don’t make strong product decisions. “Greenfield [building brand new codebases] is so easy for agents. Brownfield [writing, updating, or maintaining existing code] we all know is where the actual challenge lies.”
Replit takes a bit of a different tack: While the company has “gone very agentic,” they’ve been more conservative with AI coding, explained Amol Jain, head of product engineering. An agent reviews each pull request (PR) and assigns it a risk score; low-risk PRs are self-merged by their author, while others go to human reviewers who read the code and give feedback.
“The idea was human on the loop, not human in the loop,” Jain said. Replit’s internal tool is essentially self-driving for software engineers; devs give a task to agents, which do end to end planning, implementation, and testing.
“It’s a fleet of agents that run in their own cloud virtual machines (VMs) with access controls behind token proxies so they’re secure,” Jain said.
He shared one example where an engineer couldn’t repro or solve a “very gnarly bug” deep in its systems. It was sent to an AI manager agent, which told it to go to sleep. The manager agent then spun up a bunch of underlying agents that found the issue; it subsequently spun up a bunch more agents that found the fix. Six hours later, AI had a PR ready for the bug that had puzzled human engineers.
AI providers are also evolving beyond the lock-in model, as customers increasingly demand multi-model choice.
Kilo Code, for its part, supports 500-plus models in its gateway. “Your software that you’re using to do agentic engineering should be decoupled from the model that you’re using to do it,” Schario said.
For instance, Schario said companies often use expensive frontier-tier models to architect a project, then switch to a less expensive open-weight model for the rest of the work.
It’s also important to respect model provider limitations, such as when they need to work in closed or isolated environments or providers in their specific regions. “It’s factoring in what’s important to you, what limitations you’ve set, what data retention policies you’ve established, what keys you’ve brought in, what commits you might have … into that routing decision,” Schario said.
Replit, similarly, tends to have a better sense of the cost versus capability spectrum than its customers, Jain contended. “We are essentially making the decisions on users’ behalf of what model to use when, in what capacity, to minimize cost and maximize capability.”
Of course, an important consideration as AI adoption increases is runaway costs, which has led to some enterprises tracking and capping AI use through tokenmaxxing.
Concerns come from both sides, Schario said: internally and from customers. From the latter, she’s hearing, “I accidentally spent my whole AI budget for the year … so what do I do now?” In response, Schario said Kilo Code points customers to the same workflow: use expensive models for planning, then open-weight models for affordability.
Further, sharing skills, strong guidance, and Model Context Protocol (MCP) will empower models. “Realizing where you can really uplevel your team to help them get the most out of the models they’re using is going to make a big difference,” Schario said.
Internally, meanwhile, Schario noted one particular engineer that has a “heavy foot” and is constantly at the top of the usage board. “I regularly have to nudge, ‘What are you doing there?'” she said. It’s easy to look at a $600 bill for daily work and react, “Wow, that’s so much,” but looking at the amount of work completed can sometimes justify the cost.
“Cost per pull request is the metric that I’m paying attention to right now,” Schario said. “It feels like the closest proximity for how I can measure value.” Ultimately, AI changes how enterprises are thinking about ROI because spend is not the problem. “The spend with no return on that spend is the problem.”
Symbotic, for its part, has set per-month cost tiers for its employees. The company built a tool that gives managers visibility into PRs and usage trends. They can then move users up or down a tier as they see fit, Go explained. “Having a cap and seeing how many people went up in cap this month makes a big difference when you’re trying to corral these costs and make things efficient,” Go said.
When Cursor — which Symbotic uses heavily — ended a legacy discount that had grandfathered the company into a flat per-request rate even for frontier models, and moved everyone to full pricing, it forced a company-wide reckoning on efficiency, Go said. “People were saying, ‘You should try this model … This works better for this C# code, this whatever,'” he said.
But the cost problem is increasingly moving out of IT; Replit, for one, broadened agents beyond engineering, and eventually found that a user on the support side had “blown through an insane amount of money,” Jain said. When they looked under the hood, they figured out it was because they were running an automation on GPT 5.5 Pro Max.
“At least till that point, the ROI was rather clear,” Jain said. “We could see engineering productivity 3X, so no one had questioned it yet.”
Visibility that isn’t “anti-productive,” model routing, and sensible defaults are critical, he emphasized. “Most tasks do not need the frontier.”
Enterprise teams building AI agents keep hitting the same wall: a chatbot that can answer a prompt but can’t remember what the last five people asked it, and can’t tell you whether last month’s version actually worked.
In a fireside chat with VentureBeat’s Sam Witteveen at VB Transform 2026, Asana’s chief product officer, Arnab Bose, unpacked how his team tackled this problem to build a new operating system: Agentic Work Management (AWM). The product treats AI agents as coachable teammates that operate alongside humans rather than as one-to-one assistants.
For product builders and developers trying to move beyond basic integrations, Bose provided a look under the hood. He detailed how Asana engineered AWM, offering a blueprint for solving real-world bottlenecks and building agentic systems at scale.
To build an operating system for human-agent teams, Asana needed a ready-made enterprise context graph. They built AWM on top of their 18-year-old architecture: the Work Graph.
This graph-based database organizes information through a structure the company calls the Pyramid of Clarity. The smallest unit of work is a task with an assignee and a due date. Tasks belong to projects, projects roll up into portfolios, and portfolios connect to company-wide goals. The graph can help trace for example how a delayed design task impacts a corporate revenue goal. The Work Graph provides a real-time ledger of who does what, by when, and why.
AWM leverages this architecture to create a multiplayer teammate. A standard AI copilot is stateless and tied to a single user’s prompt. Because AWM plugs into the Work Graph, the AI can view overarching company goals, update project statuses, and share memory with human colleagues.
“Because [the agent] is plugged into the Work Graph, it’s not just looking at a particular prompt that you’re sending it or looking at a particular individual’s markdown file system on their local file,” Bose said. “It’s working off of that shared ledger for the whole company.”
AWM is already in production. Bose said Asana has “several customers live and successful on it,” including FedEx, which published its own case study on the shift.
Shipping AWM to enterprise customers required Asana to solve several technical hurdles. The first was data governance. If an AI teammate acts across a company, it builds a shared memory by learning from workflows and human feedback.
Bose highlighted a critical boundary problem: If an executive uses AWM to build workflows for a confidential project, the system must ensure the agent’s updated memory does not leak context to an unauthorized employee who interacts with the same agent later.
“[I] shouldn’t be able to leverage that shared memory when I run the AI teammate if you created that memory using that same teammate on a project that is, let’s say, a secret M&A project that I don’t have access to,” Bose said. Asana engineered a system of access controls to govern what triggers the creation of a memory versus the simple execution of a task.
Second, AWM handles dynamic model routing to abstract prompt engineering away from the user. When a user assigns a task to an AI teammate (i.e., drafting a job description for a general manager role), the AI cross-references public job postings, Asana’s internal style guide, and product requirement documents. For a complex task, the system automatically routes the prompt to a heavy frontier model — Bose pointed to Anthropic’s Opus and OpenAI’s models as examples — while lighter tasks get down-leveled to something faster and cheaper.
“We don’t want the knowledge worker to have to think through what the best possible prompt, context engineering, and attachments are that they should put into the task,” Bose said. “It should feel as if you were assigning the task to a human being.”
This dynamic routing introduces a third challenge: billing abstraction. Agentic tasks vary in computational complexity, making credit burn rates unpredictable.
“We don’t want to get into a state where our customers are having to reason about the fact that some of these tasks… are way more complex than others and they’ll be burning credits at different rates,” Bose said, adding that unpredictable pricing risked customers throttling their own employees by capping how often they could run an AI teammate.
To make AWM commercially viable, Asana designed its billing architecture to charge a static cost per task completion. The platform absorbs the complexity of model selection, token counts, and run limits to ensure predictable enterprise pricing.
AWM targets a specific problem with current enterprise AI deployments: statelessness. Developers can easily connect large language models to enterprise tools like Slack, Google Drive, or Databricks using Model Context Protocol (MCP) integrations. However, basic chat-based agents lack persistence.
Bose detailed a scenario where a user asks a chat agent to draft a marketing campaign based on historical performance and competitive research. The agent fetches data from external tools to answer the prompt, but the execution happens in a vacuum. It is a one-off task that benefits a single individual. It fails to create a reusable workflow for the next person building a similar campaign.
“The challenge with that is that those calls are stateless, and they are not leveraging a shared company brain that is this graph-based database or a context graph,” Bose said.
AWM solves this by creating a permanent state. When an AI teammate inside AWM completes a task, the system records the metadata. It registers whether the completion improved the project status and how it moved higher-level company goals.
Cloud provider CoreWeave is an early adopter using AWM to overhaul complex new product launches.
“CoreWeave is using both our deterministic AI studio workflow rules as well as multiple AI teammates to do new product launches,” Bose shared.
In the past, CoreWeave product managers filled out complicated forms detailing infrastructure, parameters, and costs. Human reviewers manually evaluated these forms and broke them out into specific tasks for finance, marketing, and hardware teams.
Under the AWM workflow, a product manager writes a standard Google document pointing to their product requirement documents. A deterministic AI workflow reads the document, automatically creates the project structure, and assigns tasks. Specialized agents then take over the execution. One agent then watches overall project status and flags bottlenecks; another, working inside individual tasks, forecasts infrastructure costs and recommends approvals when the numbers align with historical budgets. The system automatically triages the busywork while human beings focus on evaluating the AI’s outputs.
The dynamic gets complicated by the fact that the same frontier-model providers powering AWM under the hood — Anthropic, OpenAI — are also shipping their own competing agent products, like Anthropic’s Claude in Slack (Tag). Pressed on the overlap, Bose didn’t dispute the tension.
“I think that’s the reality that we all have to live in,” he said.
His case for AWM’s staying power rests on Asana’s 18 years of user-experience and workflow data, and prebuilt standard operating procedures for specific industries — expertise he argues raw frontier models don’t have. A product like Tag can work well in Slack, he said, but it requires a highly curated channel and its own separate credentials for every downstream app it touches.
“There’s a big difference between the power of the model plus a lightweight way to demonstrate its value, and something that’s pre-built … for true end-to-end use,” Bose said.
Presented by NTT DATA AIVista
At VB Transform 2026, NTT DATA AIVista CEO Bratin Saha joined VentureBeat CEO and editor-in-chief Matt Marshall to discuss the last-mile challenge of operationalizing frontier models in regulated production, where reliability, context, guardrails, and security determine whether AI delivers enterprise value. The conversation centered around the question facing every enterprise now pouring money into AI: how to convert that spending into real, tangible value.
“It’s not just a model, you’re building a system around the model,” Saha said. The last mile is the work of wrapping a frontier model in an enterprise’s own data, workflows, and guardrails.
In the end, regulated production turns on more than just technology, Saha said. Today, most enterprise AI projects fail during implementation because of poor integration, domain specialization gaps, lack of governance, and unclear ownership of outcomes. Last-mile specialization turns a capable foundation model into an enterprise agent shaped by domain-specific workflows, risk appetite, client classifications, regulatory interpretations, and institutional knowledge.
Frontier models fall well short of production-grade accuracy on many real-world insurance workflows, Saha said, but last-mile specialization can lift them to the reliability enterprises need. Out of the box, those models struggle with the complexity of regulated workflows such as multinational insurance claims.
“These forms are pretty complex, often have handwriting, lots of checkboxes, and so on,” he said, and that complexity is why frontier models like Fable 5, Opus 4.8, and GPT-5.5 fall short out of the box.
Saha said the biggest gains come from specializing the entire AI system, not just the foundation model.
That system gets specialized with the customer’s data, workflow and, in many cases, the tribal knowledge that never made it into an operating procedure document.
“The biggest bang for the buck comes from the specialization and then these specialized guardrails,” he said.
The work has three components:
capturing the enterprise’s context and making it consumable by AI
running an ensemble of models so cost does not go through the roof
and adding specialized guardrails that check the model and force a redo when it gets something wrong.
None of this involves fine-tuning. VentureBeat’s latest enterprise survey found it ranked last among companies’ model-selection priorities.
Instead, the last mile centers on domain knowledge and undocumented workflows that companies would never expose publicly without losing their competitive edge.
“The last mile is about taking data that’s proprietary to you and using that to build a system around the model that can steer the model in the right way that can put the appropriate guardrails around it,” Saha said.
In the end, enterprise AI is about moving a workflow from point A to point B rather than deploying a technology, and NTT’s advantage comes from pairing AI experts with subject domain experts.
“The only reason is because we go and talk to those human workers and we say, ‘How do you actually do the work,'” he said. That expertise is then encoded into an agent.
Success in insurance, manufacturing, and other regulated industries relies on three things at once, he added.
“You need technology, you need the domain expertise, and you need the change management expertise,” he explained, adding that across his team’s clients, technology is not the bottleneck.
For enterprises weighing large AI budgets, Saha’s said the payoff comes not from the model but from the work built around it.
“When you’re deploying AI in the enterprise, you’re not deploying a technology,” he said. “You are taking a workflow that exists and taking it from point A to point B.” The value is created by the workflow that gets moved, not the model that helps move it.
That reorders where money should go.
“Technology is not the bottleneck,” Saha said, pointing instead to the domain expertise and change management wrapped around the model, and to the discipline of commiting to all three together. Spending aimed only at the model leaves most of the return on the table.
Enterprises don’t have to choose between embedding AI into existing workflows and redesigning those workflows from scratch. NTT sees the two as successive stages of the same journey.
“We are starting with embedding in the workflow because it’s easier change management,” he said, noting that customers running mission-critical operations will not let a vendor rip out a working process midstream. “Once that happens, then we go into, how can we now reimagine this? And that really is where the biggest bang is.”
Keeping intelligence in the surrounding system rather than the model also preserves swappability and lets enterprises take advantage of open-weight and open-source models as they mature. Saha’s team runs an ensemble that mixes frontier and open-source models, and he expects the industry to lean on open weights wherever the cost of a mistake is low while reserving frontier reasoning for the cases that demand it.
“In many situations, especially in regulated industries where mistakes are very expensive, that last extra couple of percent matters,” he said.
The platform follows the same pattern: Guardrail generation and neurosymbolic models scale across customers, while capturing each organization’s tribal knowledge remains bespoke. Saha pointed to NTT DATA’s position as one of the world’s largest insurance third-party administrators as an advantage in acquiring that expertise.
“The ability to take that knowledge and trust that has been built over 20 years is very hard to replicate instantly, and I do think that is a durable aspect of what we have,” he said.
Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.
If you have built anything with retrieval-augmented generation (RAG) in the last two years, you have lived its central frustration: You chop your documents into chunks, embed them, retrieve the top few that look similar to the question, and hand them t…
If you ask an AI coding agent to write a standalone Python script to parse a single JSON file, it will likely give you a perfect answer in seconds. But the same agent often breaks if you ask it to build a systematic data processing pipeline, like ingesting thousands of messy documents, chunking text, scoring quality, and filtering noise for a Retrieval-Augmented Generation (RAG) system that fits your specific enterprise stack.
While large language models (LLMs) excel at one-off code generation, their outputs for complex data-processing tasks are typically free-form, disposable scripts. These scripts are detached from the governable workflow abstractions that MLOps teams rely on for production, making them difficult to audit or edit visually.
To address this, researchers at Peking University, Zhongguancun Academy, and Shanghai’s Institute for Advanced Algorithms Research introduced DataFlow-Harness, an open-source framework that guides an LLM agent to build structured, visual data-processing workflows step-by-step, rather than writing raw code from scratch.
The framework makes AI-generated pipelines easier to manage and integrate into existing architectures because the generated artifacts are persistent and easily editable.
The researchers report that the platform achieves a 93.3% observed end-to-end pass rate on a 12-task data-engineering benchmark. Compared to standard Claude Code, it reduces API costs by up to 72.5% and response latency by 49.9%, while achieving nearly the same success rate as an AI given the entire codebase to write standard scripts. For enterprise teams, this means getting the speed of AI automation without accumulating unmanageable technical debt, ensuring that pipelines remain secure, auditable, and ready for production.
Data-centric AI requires workflows for tasks like synthetic data generation, retrieval augmentation, and model training. While LLMs can translate natural language into executable implementations to perform these tasks, high task accuracy is insufficient for production deployment.
“The first wall is usually not writing Python,” Runming He, first author of the DataFlow-Harness paper, told VentureBeat. “Modern coding agents can often produce a plausible script quickly. The harder problem is grounding that script in a live production platform: using operators that are actually installed, matching the real dataset schema, referring to registered datasets and model services, preserving dependencies between stages, and leaving behind an artifact that another engineer can understand and revise.”
General-purpose AI agents frequently hallucinate dependencies, relying on unavailable operators or outdated platform assumptions. Instead of leaving behind an artifact that another engineer can understand and revise, they generate disposable code that is difficult to audit through workflow managing tools.
The researchers define this challenge as the “NL2Pipeline gap”: the disconnect between a user expressing workflow requirements in natural language and the production environment requiring structured and persistent pipeline assets.
The researchers demonstrated this gap in their experiments. For example, when Claude Code was allowed to write standard, free-form scripts using codebase context, it hit a 94.2% success rate. However, when restricted to only using the platform’s specific building blocks to create a native workflow graph, its success rate dropped to 83.3%. This gap is the paper’s central finding: native, governable pipelines are meaningfully harder for the agent to produce than throwaway code.
“Closing this gap requires more than improving code-generation accuracy: construction must remain grounded in platform semantics and produce artifacts that integrate with the host platform,” the researchers write.
“DataFlow-Harness changes the agent’s action space,” He said. “Instead of asking the agent to emit arbitrary code, it retrieves the live operator registry and current pipeline state through MCP and applies typed, incremental changes to a persistent DAG.”
To achieve this, the platform organizes workflow synthesis around four components: the Data Pipeline Backend, the interaction layer (DataFlow-WebUI), the MCP Tools Layer, and the AI guidance layer (DataFlow-Skills).
The Data Pipeline Backend acts as the authoritative source of truth across conversational, visual, and programmatic interfaces. It represents the pipeline as a directed acyclic graph (DAG), a structured workflow map containing data sources, configured pre-built processing modules (which the researchers refer to as “operators”), and execution dependencies. Instead of generating free-form code, agents interact with this backend through “typed mutations,” like adding an operator or connecting edges.
DataFlow-Skills are markdown files that inject domain-specific knowledge into the model’s context window, guiding it on operator-selection patterns, schema inference, and assembly procedures. Rather than letting the AI guess how to assemble components, skills provide the AI with compatibility rules, teaching it how to correctly match different data formats and handle complex data structures without breaking the pipeline.
The MCP Tools Layer gives the AI access to the operator registry and current state of the data workflow. The AI proposes structured changes through the tools layer. The system validates the changes to ensure the workflow runs in a valid sequence and that every connected module speaks the same data language.
DataFlow-WebUI provides two interfaces that allow humans and AI to build the workflow together. Developers can describe workflow requirements in natural language through a conversational interface. They can also access the workflow as a graphical map in a visual DAG editor. Here, they can directly inspect the changes proposed by the AI and make modifications.
“The current implementation performs static checks against platform metadata before accepting pipeline changes,” He said. “These include checks for registered datasets, operators and model-serving references, field flow, and some invalid parameter usage, as well as structural validity. The result is visible in a graphical editor and can be revised either manually or by the agent in later turns.”
The researchers tested DataFlow-Harness on a benchmark of 12 tasks across six industrial data-processing scenarios, such as QA generation, review governance, and schema normalization. They used Claude Opus 4.7 as the backbone model in their experiments.
They compared DataFlow-Harness against three baselines:
Vanilla CC: An unconstrained coding baseline using standard Claude Code.
Context-Aware CC: An agent that has access to the DataFlow codebase in its context window.
MCP-only: An agent that has access to the DataFlow MCP tools and is instructed to generate platform-native DAGs (without access to DataFlow-Skills).
DataFlow-Harness achieved a 93.3% end-to-end pass rate, improving by 10.0 percentage points over MCP-only and beating Vanilla CC (91.7%), while being within 0.9 percentage points of Context-Aware CC (94.2%).
Importantly, it reduced API costs to $0.261 per task, a 72.5% drop compared to Vanilla CC and 42.8% compared to Context-Aware CC. In generating workflows, it was 49.9% faster than Vanilla CC and 17.6% faster than Context-Aware CC.
DataFlow-Harness proved particularly effective on complex tasks that depend on implicit domain knowledge, like QA generation. The baseline MCP-only approach frequently generated structurally valid DAGs but struggled to infer task-specific procedures from operator descriptions alone.
To show how this works in the real world, the researchers detailed a textbook-to-VQA extraction task. This job required the AI to stitch together capabilities such as PDF parsing, layout recovery, OCR, figure extraction, multimodal understanding, and long-range question-answer matching. DataFlow-Harness achieved 97.2% precision and an 87.3% coverage rate, easily beating the baselines. By having the AI snap together existing platform assets rather than coding complex tasks from scratch, it recovered more valid QA pairs from the document.
Their experiments also showed that DataFlow-Harness is highly effective at creating data generation pipelines. For example, in a synthetic instruction-data generation task, the agent built a multi-stage pipeline that generated candidate instruction–response pairs, critiqued and rewrote them, scored them with an LLM-based judge, and filtered low-quality outputs before training.
“Such workflows are costly to build and fragile to maintain as collections of ad hoc scripts,” He said. “The harness does not make them automatically safe, but it turns them into explicit, editable stages that engineers can inspect, test, and govern using normal production controls.”
Similarly, when tasked with building a math data cleaning-and-synthesis pipeline, the data produced by the DataFlow-Harness pipeline trained a better-performing model with higher average accuracy on AIME24 and AIME25 benchmarks than the data produced by the vanilla Claude Code pipeline.
For engineering teams evaluating DataFlow-Harness, it is important to understand how it fits into existing infrastructure. Released under the Apache 2.0 license, the current implementation requires a bit of engineering to fit into popular tech stacks.
“The current implementation is native to the DataFlow platform; it is not a turnkey Airflow, Prefect, or Spark plug-in,” He said. To use those systems as an execution backbone, teams must build an adapter to connect their organization’s registry, metadata, and execution interfaces to the agent’s control layer.
Furthermore, organizations must invest in the boundaries they want the AI to respect. This requires maintaining an operator registry, defining schemas, and encoding recurring domain procedures as Skills. Because of this overhead, He recommends against using the framework for small, one-off transformations where a simple script suffices, or in legacy environments that cannot expose reliable metadata.
Finally, while the platform prevents illogical connections by validating structural properties, it is an engineering control layer, not a compliance substitute. “The harness should still be treated as an engineering control layer, not as a substitute for compliance policy, validated detection models, access controls, audit logging, or human approval,” He said.
The platform is open-source, and developers can access the source code and codebase documentation directly via the project’s GitHub repository.
As protocols like MCP become standardized, the boundary between human engineers and AI agents will shift. “The goal is not autonomous data engineering without oversight,” He said. “It is a better division of labor: agents perform repetitive construction inside explicit boundaries, while engineers remain responsible for the semantics, policies, and consequential decisions that require domain accountability.”
Enterprise AI agents can do the work — but the infrastructure to let them talk to each other, prove they should be trusted, and be audited when something goes wrong is still being built.
Here’s a look at how five startups are tackling that gap — around orchestration, observability, connectivity, and security — as shown at VB Transform 2026.
In the very near future, agents will be deployed everywhere, and they will do work on our behalf, noted Vlad Luzin, CTO and co-founder of BAND.
As he describes it: They will receive tasks, visit registries, recruit other agents to help them, delegate subtasks to AI peers in a “conversational space,” gather and share results, then return a summary to the human user.
BAND is building a coordination infrastructure layer for multi-agent AI systems to make this a reality.
Why don’t Telegram, Slack, or Discord solve the problem? These platforms were built for humans, Luzin noted. Agents have to be onboarded manually in numerous steps, and they can’t see each other; “they are still alone in a kind of digital solitary confinement.”
Similarly, Claude is stateless, and devs often have multiple sessions open at a time that they toggle between for different tasks — something Luzin said creates real friction.
The challenge is connecting remote processes, which Luzin called a distributed systems problem.
“The transportation layer needs to be solved first, how the agents communicate in real time,” he said. Conversations can’t happen through IPs and URLs; they need to be bumped to the abstraction layer so agents can talk across channels, conversational spaces, and platforms.
“Agents see each other. They understand. They can collaborate together. They discuss issues. They fix issues, and they ask for review from another,” Luzin said.
BAND supports autonomous workflows that can run for eight to 20 hours and is compatible with A2A and MCP protocols, according to Luzin. Importantly, humans can join the conversation as agents converse and discover one another, he said.
“We can record and show you all the tasks that your agent generates in real time,” Luzin said.
The biggest challenge defenders face today is that they’re still running at human speed, but adversaries are running at machine speed, said Tom Findling, CEO and co-founder of Conifers.
Attackers are already adopting agents, Findling said, and they only have to be successful once to penetrate an enterprise. Malicious campaigns that used to take months and weeks now take hours, even minutes. Security operations, on the other hand, are fragmented, manual, inefficient, and slow.
Findling said Conifers has taken various components of cyber defense — private intelligence, hunting, detection, engineering, investigation, response — and made them agentic. They then broke down the silos between them, he said. Various agentic systems can communicate with one another to ensure that operational defense and active defense are always on and adapting.
Findling said that Conifers’ system is condensing containment time from 7 hours to 12 minutes, and that the company can turn around complex cyber investigations in four minutes or less.
He emphasized the importance of connecting to an enterprise’s existing security tools, whether that be endpoint detection and response (EDR), security information and event management (SIEM), posture management, or others. Conifers helps customers understand their security posture, pain points, which controls are working and which are not, and the areas to invest for the best ROI.
“The threat landscape is changing, detection stays the same, and threat intelligence is not being operationalized,” Findling said. “This is a job for agents.”
One of the defining problems of the current era is finding critical issues in AI agents, says Ben Hylak, CTO of Raindrop AI.
It’s what he called a “double whammy”: As agents become more capable, complexity increases, as do timelines; they are running for hours or days in some cases. Secondly, issues become catastrophic in sectors like healthcare or defense.
“This problem is getting a lot worse as models and agents improve,” Hylak said, “and I think there’s good reason to believe it will continue to get worse.”
Raindrop AI’s platform finds critical issues in agents in production and simulates fixes based on past user behavior, Hylak said. That lets teams confirm a fix works as intended before it’s live, without introducing unexpected side effects.
The startup’s reinforcement learning (RL) platform optimizes harnesses and trains models directly from Raindrop data, he said. Its pre-deployment simulation engine helps identify what fixes would actually impact in production; its live A/B testing then shows those changes in action.
Messages, tool calls, retries, and errors are captured in one place, and human users are notified (typically via Slack) when there’s an issue, he said. Models are trained for every customer, and signals are powering continual learning across models and harnesses. “It is condensed into something that is actually navigable, easy to understand, easy to verify,” Hylak said.
AI agents are designed to do all kinds of things for you, but they often hit three major snags: authorization, governance, and reliability.
To act on behalf of real users with real permissions, agents need a new type of security architecture, said Sam Partee, co-founder and CTO of Arcade.dev.
Partee said his company’s secure agent runtime provides this authentication and authorization layer so agents can pass critical security reviews. It also provides observability so human users can watch everything an agent is doing. Actions are attributable to the exact moment in time with the least amount of privileged scopes.
Arcade is available in an installable plugin that can be deployed on-prem in a clean room-like environment; companies can continue to use their own sign-in and security tools, Partee said. Whenever anything is run in Arcade, it’s gated by the same role-based access controls (RBACs), intrusion detection and prevention systems (IDPS), policies, entitlements, and other already-established checkpoints.
Arcade is tackling the supply chain attack problem, which has “gotten so rampant; it’s unbelievable,” Partee noted. Security and observability have continued to be challenging because “largely, the abstraction has been wrong.”
Solving enterprise customer experience (CX) is “really not straightforward,” said Claudio Rodrigues, CPO of Omilia.
Heuristic-based systems are controlled but slow; agentic systems are fast but unpredictable, Rodrigues said. Omilia built its platform to deliver both control and speed together.
The agentic, self-learning offering is built on a philosophy of observing customer service operations as they actually happen, rather than in the abstract. Omilia’s agents observe problems first-hand, listen to every customer and agent interaction, ingest data, API specs, screen recordings, and standard operating procedures (SOP), then map those to use cases for customer support, he said.
Contact centers should be a revenue driver, Rodrigues said, and Omilia’s differentiator is its speech-to-text systems and governance and observability layers.
AI creates insights, suggests improvements, automatically generates conversational agents, pulls information from documents and APIs, and designs dialogue flows. Human experts can then test real and simulated interactions and deploy into production under their supervision. Omilia combines all of this into one enterprise-wide engine that continuously learns over time, Rodrigues said.
Rodrigues said the company handles more than 3 billion calls a year, 1 million-plus voice calls a day in some deployments, and has seen 30 to 45% improvement in time to resolution (TTR). Omilia’s agents generate 21x more upsell revenue versus human agents, he said.
In a mature deployment, automation “easily” reaches 80 to 90%, he said. However, “human in the loop is still very fundamental for us.”
Nimble, a New York City-based tech startup VentureBeat previously covered for its efforts to re-invent web search for enterprises by using multiple AI agents to improve accuracy and depth, is taking another step toward its vision of a world in which agents do most of the web searching instead of us typing and reviewing the results manually.
Nimble today launched Web Search Agents, a new retrieval system designed to help AI agents perform more 21% more accurate web research while using significantly fewer tokens — 51% less compared with leading AI search alternatives on comparable, according to the firm.
While Nimble did not disclose its specific benchmarking methodology or competitors evaluated, the results underscore a growing trend in enterprise AI: optimizing retrieval has become as important as improving the underlying language models themselves.
Nimble’s leadership says the product combines self-learning retrieval strategies, proprietary web indexes, and live web access to deliver domain-specific search capabilities that outperform general-purpose web search services for enterprise workloads.
“Our research team built self-learning retrieval algorithms that learn a customer’s domain,” said Nimble CEO and co-founder Uri Knorovich in an interview with VentureBeat. “They find the exact information more efficiently, reduce the amount of multi-hop reasoning required, and lower token usage while improving accuracy.”
Rather than positioning itself as another general search engine, Nimble is targeting developers building autonomous agents that require continuously updated information from the public web for research, lead generation, competitive intelligence, compliance, and other business-critical workflows.
It’s also designed to slot in seamlessly to an enterprise’s existing systems and workflows.
“You can run the agent directly through the Nimble API with zero infrastructure,” Knorovich said. “For large enterprises, we’re partnering with Microsoft, Oracle, Snowflake, and others so customers can deploy these agent systems inside their own infrastructure.”
How does it work and stack up to other, existing AI-powered search and agentic systems? Read on to find out.
Most AI applications today rely on general-purpose search application programming interfaces (APIs) for search engines and public knowledge bases that return broad collections of files, leaving the language model responsible for determining which sources are relevant.
That process often requires multiple retrieval steps, additional reasoning, and significant token expenditure before an agent produces an answer. This is obviously inefficient and raises the cost spent to run AI search looking through irrelevant sources.
Nimble argues that before long, every enterprise will need its own methods for searching, retrieving, and validating external information since each enterprise relies on its own distinct preferred sources, signals, and standards of trust.
As such, instead of applying one search strategy to every workload, Nimble’s Web Search Agents are designed to learn the characteristics of a specific domain and adapt how information is retrieved, providing agents with structured, relevant context rather than forcing them to sift through large amounts of generic search results.
“Instead of one generic retrieval model, we build specialized retrieval models for each customer’s domain, making them faster, cheaper, and more accurate,” Knorovich explained. “A single enterprise can run hundreds of different agents. Each one has its own domain expertise, guardrails, goals, and search algorithm. The optimization starts with the second search, without requiring any setup from the customer.”
Its goal is not only to reduce redundant retrieval, but also to shorten multi-step research paths and avoid repeatedly sending raw pages through a language model for parsing, resulting in the 51% reduced token figure the company cites.
The distinction is particularly relevant for long-running enterprise agents performing research over hours or days rather than answering simple consumer questions. In those scenarios, reducing unnecessary tool calls can significantly lower operating costs while improving answer consistency.
That emphasis reflects a broader shift occurring across the AI tooling ecosystem. As foundation models become increasingly capable, infrastructure vendors are competing on everything surrounding the model—including retrieval, orchestration, memory, observability, and governance.
The launch builds on Nimble’s broader strategy of becoming an enterprise web intelligence platform rather than simply a web scraping provider. Earlier this year, the company introduced its broader Agentic Search Platform following a $47 million Series B financing, positioning itself as infrastructure that transforms the live web into structured, machine-readable data for AI systems.
The company’s latest release extends that vision with a concept it calls “Harness as a Tool,” which powers its new domain-specialized Web Search Agents. Rather than requiring engineering teams to assemble separate search APIs, browser automation, extraction pipelines, validation logic, memory systems, and orchestration code, Nimble packages those capabilities behind a managed interface.
The harness can determine what to search, navigate pages when conventional indexes are insufficient, extract relevant information, validate the results, and return the final context in a form designed for downstream agents.
Nimble also says the system retains domain-specific memory and builds proprietary indexes that improve as customers run more searches.
“The biggest research breakthrough is adding semantic memory and a caching layer to the agent,” Knorovich told VentureBeat. “The agent learns usage patterns and domain expertise over time, so every subsequent search becomes faster and more efficient.”
As for what domains Nimble can tackle, the company says it can address virtually any knowledge work domain.
“We’ve seen customers build investment banking analysts, competitive intelligence agents for product managers, go-to-market research agents, newsroom monitoring, insurance applications, life sciences research, and supply chain optimization,” Knorovich said. “Our customers surprise us every day with new agent use cases.”
However, for enterprises concerned about data privacy and retention, Knorovich assured VentureBeat that: “Nimble is zero-data-retention by design. Customer queries are never stored in our environment, and when customers deploy semantic memory and self-learning models, that knowledge stays in their own tenant—not ours.”
Nimble supported the announcement with early customer examples from AI-native software vendors and enterprise users.
AI-native CRM company Rox reported achieving a 20× reduction in token costs after adopting Nimble’s retrieval infrastructure while simultaneously improving the quality and completeness of information available to its AI agents.
Although the company did not disclose detailed workload measurements or a reproducible baseline, the example illustrates the operational savings retrieval optimization can provide for high-volume agent deployments.
Nimble says its infrastructure currently supports more than 90 million searches each day across Fortune 500 enterprises and AI-native companies operating mission-critical workflows where accuracy, completeness, and enterprise control are essential.
The platform is immediately available through an API, SDK, and Model Context Protocol (MCP) integration, allowing developers to connect Nimble directly into AI agents regardless of the orchestration framework they use.
Developers can use the platform for several categories of web intelligence, including:
Low-latency live web search
Deep multi-step web research
Web crawling
Structured dataset generation
Domain-specific information retrieval
The company also provides documentation and pre-built agents for common web extraction tasks while allowing developers to build custom retrieval agents using natural-language descriptions instead of manually maintaining scraping logic.
Nimble is offering two notably different consumption models. Developers can begin with a pay-as-you-go Agent API priced from $0.025 per Web Search Agent request at the listed low-effort setting. Companies that want Nimble to configure and manage custom data delivery can instead buy annual managed plans beginning at $2,500 per month.
Nimble enters a market that has rapidly expanded beyond traditional web search into autonomous research agents capable of planning, browsing, reasoning, and synthesizing information. Products such as ChatGPT Deep Research, Google Gemini Deep Research, Alibaba’s Tongyi DeepResearch, Perplexity, and Sakana Marlin all seek to automate knowledge work that previously required hours—or, in Marlin’s case, potentially weeks—of human research.
Rather than competing head-to-head as another end-user research assistant, however, Nimble is positioning itself one layer lower in the AI stack—as the web intelligence infrastructure that powers those agents or custom enterprise applications built on leading foundation models.
That distinction reflects an increasingly important architectural shift in enterprise AI. Most “Deep Research” systems optimize the overall research workflow, generating search plans, iteratively gathering information, and producing synthesized reports.
Nimble instead argues that the retrieval layer itself has become the primary bottleneck for enterprise AI deployments. If an agent retrieves too many irrelevant pages or performs unnecessary search iterations, token consumption, latency, and operating costs all increase before the model even begins its main reasoning process.
“Customers across life sciences, insurance, healthcare, pharma, retail, and digital-native companies are all telling us the same thing: we need to feed our agents with more accurate context, and we need to reduce the amount of tokens every task consumes,” Knorovich said.
The launch blog makes that argument more concrete by describing how teams frequently rebuild the same retrieval stack themselves. A production agent may start with a search API, then accumulate browser controls, parsers, extraction components, validation steps, memory, caching, evaluations, and custom workflow logic. Nimble is positioning its harness as a managed alternative to that growing engineering burden.
In Nimble’s view, improving retrieval before reasoning begins is more valuable than simply giving a language model more documents to analyze. The company’s Web Search Agents therefore adapt retrieval strategies to a particular workload, combining proprietary indexes with real-time web retrieval and task-specific search policies rather than applying the same search algorithm across every domain.
That makes Nimble less of a direct competitor to OpenAI’s or Google’s research assistants than to developer-focused retrieval infrastructure such as Exa and Tavily. Those platforms also provide AI-native search APIs and research capabilities, but Nimble differentiates itself by emphasizing self-learning retrieval strategies, proprietary indexing, enterprise governance, managed delivery, and token efficiency for production agents.
For organizations building their own AI systems, the distinction could become increasingly important. Foundation models are becoming more capable across the industry, shifting competitive differentiation toward the infrastructure surrounding them—including retrieval, orchestration, memory, observability, and governance. Nimble’s strategy reflects that broader trend, betting that better web intelligence can deliver larger operational gains than incremental improvements in model reasoning alone.
The different positioning is also reflected in pricing.While consumer-facing AI research assistants are generally sold as productivity subscriptions for individual users or teams, Nimble is pricing its managed service as enterprise infrastructure designed to power production applications. Its pay-as-you-go API, however, gives developers a lower-cost path to test the underlying agent technology before committing to a managed deployment.
|
Platform |
Primary audience |
Primary focus |
Lowest publicly available price (USD) |
|
Nimble |
Developers and enterprises |
Managed web retrieval and orchestration infrastructure combining specialized search, browsing, extraction, validation, proprietary indexing, and memory |
$0.025 per Agent API request (low-effort setting). Managed service starts at $2,500/month (Startup plan, billed annually). |
|
ChatGPT Deep Research |
Professionals, enterprises, and knowledge workers |
Autonomous multi-step research with iterative browsing, synthesis, and citations |
$20/month (ChatGPT Plus). Higher limits are available with Pro, Team, Enterprise, and Edu plans. |
|
Google Gemini Deep Research |
Consumers and enterprises |
Research planning integrated with Gemini, Google Search, and Google’s productivity ecosystem |
$19.99/month (Google AI Pro, U.S.). Higher-capacity AI Ultra and enterprise Workspace offerings are also available. |
|
Tongyi DeepResearch |
Developers and AI researchers |
Open research model for long-horizon information-seeking and agentic search |
Free (open source). Users are responsible for their own infrastructure and cloud compute costs. |
|
Perplexity |
Consumers, professionals, and enterprise teams |
AI-powered web search and cited research |
Free entry tier. Perplexity Pro starts at $20/month with Enterprise Pro available separately. |
|
Exa |
Developers and AI platform builders |
AI-native search, content retrieval, and asynchronous research agents |
Free developer tier (includes monthly credits). Paid Search API pricing starts at approximately $7 per 1,000 requests while Agent runs range from $0.012 to $1.00 per run depending on effort level. |
|
Tavily |
Developers building AI agents |
Search, extraction, crawling, and research APIs for agents and RAG workflows |
Free developer tier (1,000 monthly credits). Pay-as-you-go usage starts at approximately $0.008 per credit. |
|
Sakana Marlin |
Enterprises, strategy teams, financial institutions, and research organizations |
Ultra Deep Research for hours-long strategic reasoning and executive-grade reports |
Pay-as-you-go from approximately $0.61 per credit (¥98/credit) with with 100 credits required per research run (approx $61 per run). The first subscription tier is Pro at approximately $936/month (¥150,000/month) followed by Team at approximately $2,495/month (¥400,000/month) with Enterprise pricing available by quote. |
The comparison reveals three increasingly distinct markets.
ChatGPT Deep Research, Gemini Deep Research, and Perplexity operate primarily as user-facing research assistants.
Exa and Tavily provide developer-facing retrieval and research APIs.
Nimble and Sakana Marlin occupy more enterprise-oriented territory, but at different layers: Nimble supplies retrieval infrastructure, while Marlin performs long-horizon strategic analysis.
Sakana Marlin is particularly useful as a counterpoint. It is positioned as a “Virtual CSO” rather than a search API, running autonomous research loops for as long as eight hours and producing executive-ready reports, references, and supporting materials.
Nimble, by contrast, is designed to sit beneath those kinds of systems, supplying the specialized retrieval, browsing, extraction, validation, and orchestration that enterprise agents need to gather reliable external information before reasoning begins.
The comparison therefore should not be read as a direct price-to-price evaluation. A $20/month ChatGPT Plus or $19.99/month Google AI Pro subscription buys an individual AI workspace with Deep Research capabilities.
Nimble’s $2,500/month managed plan funds concurrent production agents, managed ETL, MCP integration, web-page capacity, storage, and hands-free data delivery.
Sakana Marlin’s approximately $936/month (¥150,000/month) Pro plan pays for extended, compute-intensive strategic research workflows.
Each price reflects a fundamentally different product boundary and deployment model rather than simply a different level of AI capability.
As enterprise AI systems mature, the industry is increasingly recognizing that model quality alone does not determine application performance.
Large language models frequently fail not because they cannot reason, but because they lack timely, trustworthy external information. That reality has fueled rapid investment across retrieval-augmented generation, AI-native search, web intelligence platforms, knowledge graphs, browser automation, and agent infrastructure.
Nimble’s launch reflects this evolution by focusing less on building another frontier model and more on improving the quality of information flowing into existing ones.
Whether the company’s reported 21-point improvement in answer quality and 51% reduction in token usage hold up across a broad range of enterprise deployments remains to be independently validated.
The larger strategic bet is that, as frontier models become more interchangeable, companies will differentiate themselves through the data, retrieval policies, trusted-source rules, memory systems, and orchestration layers surrounding those models. Nimble is not trying to build the researcher that sits in front of the user. It is trying to become part of the infrastructure that determines what the researcher can find, how efficiently it can find it, and whether the resulting evidence is complete enough to support production decisions.
Web Search Agents are available through Nimble’s API, SDK, and MCP integrations, with a free trial available for developers evaluating the platform.
Target SVP Siobhán Mc Feeney says the AI models her company runs aren’t what gives Target its edge — everything built around them is.
“There’s a lot in it. That to us is the moat,” Mc Feeney said at VB Transform 2026. “The models are great, and they’re important. They’re just not sufficient to be the competitive advantage.”
That discipline shows up early in how Target decides whether to build an agent at all. Mc Feeney was blunt, even “controversial” by her own admission, about the current AI moment: every enterprise wants AI agents, but not everything needs one, she said.
Agents earn their autonomy over time rather than getting it by default, she said — a principle that runs through everything Target has built around them.
Mc Feeney said the goal is to make sure agents are aimed at the problems that drive the most value for Target’s guests. “We want to make sure we’re investing in the right places,” she said.
Agents are becoming part of Target’s underlying architecture, increasingly connecting signals, systems, and decisions across supply chain, replenishment, and demand forecasting.
Mc Feeney framed it as retail’s oldest promise — the right product, in the right place, at the right time — delivered at scale.
But her team has been deliberate about building AI agents, beginning with the simplest, most obvious question: What is the problem they’re trying to solve? This leads to several follow-on questions:
Does that problem need an agent?
If it does, what type of agent? An orchestrator? A super agent? A domain-specific agent?
Or is what you’re calling an “agent” actually just a tool?
“You define that upfront, and this may sound a little process-heavy, then you have to register and certify your agent,” Mc Feeney said. Because a solution may already exist, and you don’t want to duplicate work.
Agent design kicks off another series of important questions: What triggers an agent to act? Automation? An engineer? A timer? What needs to be put in place to track that?
“We’re trying to make sure we have lineage from the very beginning — the birthing of this agent, all the way through — because at 2 a.m. one morning, when something goes sideways, we want to make sure we understand everything that happened,” Mc Feeney said.
Autonomy level is another consideration; new agents typically start with base autonomy and earn more over time. What the agent has access to is a separate question: what data, what systems, what tables, what databases?
Finally, there’s monitoring and observability; agents won’t solve problems, or improve over time, if they’re not continuously evaluated.
“We measure everything: What it was intended to do, its calibration, its trajectory, not just runtime and latency,” Mc Feeney said. This creates full transparency, and allows agents to be tweaked over time.
“You’re talking about architecture and taxonomy and a data governance layer that absolutely had to be established,” she said.
There’s a lot in these “layers of autonomy” — that foundation is what gives Target the ability to scale and properly invest in the right models for the right problem.
Models have different “gradients” that are better for different jobs; for instance, frontier models excel at complex tasks that require crunching billions of pieces of data (like in heavy merchandising supply chains). But in some scenarios they can be cost-prohibitive.
“So it’s making sure there’s always a cost benefit,” Mc Feeney said.
A digital-twin simulation predicted men’s shorts inventory across three Target stores in Long Beach this summer — and one store came back needing six to seven times more stock than the others, she said. Inventory analysts’ first reaction: That can’t be right. But the system had found something they hadn’t factored in. That store sat less than two miles from the beach; the other two were 10 to 12 miles inland. Analysts let the recommendation stand, and the stock sold through.
“This is science. This is mathematically more significant and more confidence-filling than humans doing it,” Mc Feeney said. Results like that are what let Target’s agentic systems earn more autonomy over time, she said.
Target looks at AI agent autonomy as “earned” and structures it as a four-level ladder, Mc Feeney said: agents start by making observations without acting, then move to suggesting actions while waiting for approval, then to acting within defined guardrails. At the highest level Target currently operates, agents run end-to-end — but still with a human in the loop.
“The autonomy levels for the agents are super important,” Mc Feeney said. “They earn them, and they can lose them if they don’t perform as expected.” Models that drift will be taken out of service.
As she put it, humans earn autonomy when we prove we can do something over time. Nobody is given a bunch of extra responsibilities just because; they have to have shown they’re able to handle them.
In a similar way, agents can be scientifically measured and quantified: how accurate they were, how much they drifted, and how close they came to their intended goal. This helps establish guardrails, allowing builders to work faster, and “go fast forever,” because they’re not constantly wondering where the guardrails are.
“If you follow these guardrails, you [follow] security guidelines, you register the agent, and something still goes wrong, we have full lineage all the way through from the start,” Mc Feeney said. “Our ability to recover is much better.”
When it comes down to it, agent success is a confluence of factors, not just one, she said: “It’s about your architecture. It’s about your taxonomy. It’s about the autonomy levels your agents have, and it’s about security and observability.”
Even when agent autonomy is high, though, builders must still be held accountable when something goes wrong. Mc Feeney noted that teams are now working at speeds no one could have anticipated, which means evaluation harnesses have to be established and agents registered and tracked.
A lot of it is cultural; the workforce is being reshaped and builders and engineers need new skills to manage human workers and AI systems side by side. These contexts are quite different, but the career evolution is “super exciting.”
“You’re a builder. You’re observing agents building, and you’re also coaching humans observing agents building,” Mc Feeney said. “The level of nuance is pretty special.”
Instacart is posing the provocative question: What if most of the work your engineers do today should, in fact, be done by machines?
At VB Transform 2026, CTO Anirban Kundu argued that dev teams continue to waste their time on draining, repetitive, high-volume work; this should be absorbed by AI agents so that humans can focus on problems that require judgment, intent, and exception handling.
In fact, in 97% of cases, Instacart’s builders don’t even read code anymore.
“In the past, the tactical level was the creation of the code,” Kundu said. “In the most tactical level going forward, it’s going to be, ‘How do you navigate around the AI system to give you what you want?’”
That doesn’t mean humans never look at code; agents handle the bulk of code generation and boilerplate, particularly with newer projects where code is generated or regenerated on a weekly basis.
“The benefit of that is we don’t care about tech debt anymore,” Kundu said. “Things that are not active just get dropped out and then it gets rebuilt, kind of like how we used to build assembly code or object code.”
So why not 100%? The remaining 3% is in legacy, compliance, and latency-sensitive systems and workflows, or driven by a “boatload of code” that is dead, not active, or half-active. These cases still need careful human attention.
Instacart is slowly “smoothing those parts out,” however, breaking systems down in an aptly-named project Atoms, then building them back up in a cleaner, more modular form. Kundu’s team started with the “monoliths” and is shifting to remote procedure call (RPC)-driven architectures.
But evaluation remains one of the overarching challenges. Code reviews aren’t as relevant when AI is generating code — as Kundu noted, “the lines of code are going to be correct, the syntax is going to meet your expectations” — so the goal is to move to an “intent model.” That is, training devs so they can ask different models the right questions from an intent perspective.
Evals are then performed independently: Roughly 7,000 automatic evaluations run each month, and the system answers 8,000-plus real-time developer queries with about 99.9% accuracy.
Dovetailing with this, Instacart has built an agentic site reliability engineering (SRE) system trained on years of the company’s own incidents and root-cause analyses rather than generic failure data. Instead of teaching a model how production outages work in the abstract, the team fed it the specific ways Instacart’s systems have broken over time, along with the ways humans diagnosed and fixed them.
As a result, the company has seen accuracy in detecting and mitigating production issues jump from roughly 60 to more than 90%.
Kundu pointed to one example with Instacart’s internal tool Blueberry. The AI SRE colleague watches 200-some-odd Slack channels, monitors signals, and looks for patterns across human conversations and alerts.
In one incident, a database shard backed by an EBS volume that had a “hiccup” for a period of time. The human team did not immediately suspect AWS disk issues and were “obviously scrambling” to figure out why this particular shard misbehaved.
But about 20 minutes in, Blueberry posted on Slack, pointing to a specific blip and tying it to a feature-flag-like system called “roulette” that had been inadequate. “It’s supposed to be rolling out in this cadence, [but] it had been too much,” Kundu said.
Blueberry figured it out, and the team resolved the incident. “Would have a human been as quick? I think the problem is human intuition would hold us back a little bit,” Kundu said.
Humans tend to default to patterns we’ve seen before, then resort to debugging; Kundu called this the “first brain-second brain kind of thing.” But Instacart’s agentic SRE is actually “more comprehensive in its ability to look at everything and then be able to decide what does or doesn’t matter.”
Looking ahead, the most tactical work for engineers will be navigating AI systems: Designing and supervising evaluation processes; coordinating multiple simultaneous experiments and features; managing constraints like limited top-of-funnel traffic for testing; figuring out when to escalate; identifying edge cases and where things might break.
Domain expertise is also being rethought in the age of AI. Instead of bottlenecking changes through a single “owner” team that touches the code, Instacart is embedding domain knowledge into definitions and specs that any team can use.
“We’ve lived in this world where this group or this engineering team is the one that can touch the code and make the modification,” said Kundu. “We’re trying to move into a world where the code becomes completely democratized across groups.”
Software engineers at General Motors’ (GM’s) autonomous driving division spend only 15% of their time writing code, according to Rashed Haq, GM’s VP of autonomous vehicles, in a recent onstage interview at VB Transform 2026.
GM is now using AI agents to accelerate much of the other 85%—including analyzing vehicle data, triaging problems, running experiments and testing potential fixes.
The result, Haq said, is roughly three times as many merged pull requests across GM’s autonomous vehicle engineering organization, faster releases and fewer defects escaping into later stages of development.
Haq said GM achieved those gains by redesigning complete engineering workflows around agents — not merely adding an AI coding assistant.
“If you give somebody just a chatbot which can do coding, there’s still a lot of inefficiency built into that process,” Haq said during a main-stage fireside chat at the luxurious Hotel Nia in Menlo Park, California.
GM’s 15% figure may sound strikingly low, but research conducted before the rise of generative and agentic AI reached a similar conclusion.
A 2019 Microsoft study based on responses from 5,971 professional developers found that they spent an average of 96 minutes writing code on good workdays and 66 minutes on bad ones. That equals roughly 20% and 14% of an eight-hour day, respectively. A 2018 Stripe survey found that the average developer spent more than 17 hours each week on maintenance work such as debugging and refactoring.
There is no single industry benchmark: the Microsoft researchers noted that earlier studies placed coding anywhere from 9% to 61% of developers’ time, depending on how researchers defined and measured the work. But the findings reinforce Haq’s central argument. Long before agents arrived, writing code represented only one part of software engineering. Accelerating that step alone leaves much of the development process untouched.
“Doing it by loop became really important,” Haq said.
GM divided its autonomous vehicle work into several loops: developing and testing software in simulation, testing vehicles on public roads and monitoring vehicles after they reach customers. It then looked for the longest bottleneck in each loop, automated it and repeated the process.
GM connected agents to internal tools and petabytes of company data through customized Model Context Protocol (MCP) servers. It also created version-controlled “skills,” or instruction documents that tell agents how to perform specific tasks.
One high-value application involves telemetry collected from vehicles on public roads. Agents can analyze that data, conduct an initial triage and create issues for engineers to investigate. Through MCP connections, they can also call the underlying tools used by WebViz, GM’s system for visualizing vehicle telemetry, rather than relying on the same graphical interface a person sees.
The findings still need to make sense to engineers. “The output has to be human readable,” Haq said. An agent might identify a potential problem, locate the affected component, search historical data for similar incidents and provide examples supporting its conclusion.
GM bases an agent’s permissions on those of the engineer using it.
“If an engineer was going to do this task and they need access to these things, then their agent needs access to those things,” Haq said. “The engineer still is accountable for the output of the agent.”
The company also uses background agents to run machine-learning experiments in parallel. An engineer defines an experiment and its parameters, then agents execute tests and collect the results.
GM treated its internal agent platform as a product and assigned four deployed engineers to work directly with engineering teams. They helped employees identify useful workflows, spread successful practices and adopt the tools.
Haq said the resulting increase in merged pull requests represents more than higher code volume. “The velocity at which we’re releasing new features” increased, he said, while releases produced “fewer test escapes, bug escapes” and other problems.
People remain responsible at critical control points. Haq said GM established structured and unstructured tests and performance measurements before accelerating the wider workflow. Engineers review those measurements and determine whether each test still captures its intended goal before work advances toward production.
Haq said GM initially expected a more modest productivity gain. “I think our only surprise was how much we could do,” he said.
GM’s approach didn’t start with handing every developer a code generator. It started with mapping the full path from problem discovery to a verified fix in each loop — simulation, road testing, post-deployment monitoring — then giving agents controlled access to the tools and data needed to shorten the longest bottleneck at each stage.