Elastic's Agent Builder Tackles Retail AI Shopping Assistant Challenges

The race is on for retailers to meet evolving customer expectations for richer online shopping experiences, moving beyond simple product searches to interactive, personalized, and proactive AI-powered guidance. However, constructing AI systems that act as knowledgeable, fast, accurate, and cost-effective store employees presents significant challenges. Elastic has announced the general availability of its Agent Builder, a solution designed to tackle these issues by optimizing how AI shopping assistants function through advanced context engineering. This new offering aims to bridge the gap between promising AI demonstrations and the often-disappointing reality of production deployments.

Details: Addressing the Core of AI Shopping Agent Failures

Retailers investing in AI shopping assistants are frequently encountering a disconnect between initial demos and real-world performance. Common production issues include slow response times, inaccurate product recommendations (e.g., suggesting incorrect sizes), and an inability to recall customer history or product details accurately. These failures are not typically due to the underlying large language models (LLMs), which are inherently powerful. Instead, the problem lies in the agent's lack of inherent knowledge about the retailer's specific catalog, vocabulary, and business rules. Without this context, agents must rely on costly and time-consuming tool calls to discover information during each conversation, leading to increased latency and token consumption before any useful interaction can occur.

Agent Builder tackles this by pre-computing a structured context layer from existing data signals. This approach significantly cuts down the exploratory work an agent must perform, making its behavior more governed and predictable. In analogous document-retrieval scenarios, pre-computing context has demonstrated reductions in input tokens by up to 75%. Elastic anticipates comparable savings in e-commerce, given the rich signals retailers already possess, provided they are assembled in a usable format before the agent begins its reasoning process. Elastic's integrated search capabilities, including semantic, hybrid, and keyword search, alongside filtering and aggregations, position it as a vital context engine for these AI assistants.

Context: E-commerce AI vs. General RAG

Much of the existing research on reducing AI agent costs focuses on document retrieval, such as question-answering over knowledge bases. An Elastic experiment, detailed in "Cutting agent costs with pre-computed context," showed that pre-extracting structured facts from documents before an agent call could reduce input token consumption by up to 75% and improve answer accuracy from 60% to 92% on a factual benchmark. While this principle applies to e-commerce, the domain presents a fundamentally different structure than a document corpus. A product catalog is a highly structured index with specific field semantics, domain-specific vocabulary, and business rules that often override pure relevance.

The failure modes in e-commerce AI agents differ from general document retrieval augmented generation (RAG) failures. These include vocabulary mismatch, where an agent fails to map natural language queries (like "navy jumper") to canonical field values (like colour: NAVY and masterCategoryNames: "Knitwear & Jumpers"). Another issue is hallucinated filter values, where agents construct queries using invalid fields or values because they lack knowledge of the catalog's structure. Furthermore, context-free personalization leads to generic recommendations, failing to account for individual customer preferences or purchase history, which is a significant step backward from a well-tuned keyword search.

The Context Layer: Essential Signals for Smarter Agents

E-commerce is particularly well-suited for pre-computed context because retailers typically possess a wealth of relevant data signals. The primary challenge is not data scarcity but data assembly. These signals can be broadly categorized into two groups: catalog vocabulary and business policies, which are foundational and require original engineering; and live facet state, user profiles, and session history, which are valuable but more readily available. Catalog vocabulary is crucial for preventing vocabulary mismatches and hallucinated filter values by mapping natural language to exact field values and category paths. Business policies ensure recommendations align with legal requirements and trading strategies, preventing issues like age verification failures or missing specific product ranges.

Additional signals enhance the user experience. Live facet state prevents recommending filters that yield zero results or out-of-stock items for the current query. User profiles allow returning customers to avoid restating preferences like size, budget, or brand affinities. Session and purchase history helps agents avoid re-recommending dismissed items or repeating suggestions. While these latter signals are more aspirational and depend on CRM and analytics integration, they are vital for a truly personalized experience. Elastic's approach emphasizes assembling these signals into a structured context layer that the agent can consume before initiating its primary reasoning process, thereby eliminating costly, sequential tool calls.

How Context Engineering Works Before the LLM Call

The core innovation of Agent Builder lies in its pre-LLM context assembly. Instead of an agent making multiple, sequential, LLM-driven tool calls to discover information—each costing significant latency and tokens—the context is built in parallel before the LLM is invoked. An exploratory tool call can take between 600ms to 1 second. An agent performing several such calls for vocabulary, policy, and facet state could add two to three seconds to its response time before even beginning to answer. By replacing these discovery calls with a parallel fetch, the context build takes roughly the same wall time as a single LLM tool call but effectively replaces three or four.

This pre-computation also offers greater determinism and control compared to relying on the model's tool-selection choices. The token reduction is substantial: each exploratory tool call returns raw data that the model must process. A pre-assembled context summary replaces this with structured facts that the model can consume in one pass. This significantly reduces input token consumption, mirroring the up to 75% reduction observed in document search benchmarks. While the exact multiplier will vary by implementation, the principle holds true for e-commerce due to the similar exploration patterns against a structured catalog.

Governing the Context Layer Without Automated Drift

A critical distinction in Elastic's approach is the governance of the context layer. Unlike fully automated knowledge extraction systems, the context index in e-commerce requires human review, particularly for business policies. Policies governing age verification for alcohol queries or routing for specific dietary needs are not mere relevance configurations; they are trading decisions with potential legal and brand implications. An automated system generating new policies from traffic patterns without human oversight poses a compliance risk. Therefore, while signals like user profiles and session history can be more automated, the core vocabulary and business policies necessitate human authorship and governance to ensure the AI agent aligns with business intent and regulatory requirements, preventing automated drift and maintaining brand integrity.