Concurrence Scales Healthcare AI to Trillions of Tokens on Databricks

In the high-stakes world of healthcare, artificial intelligence demands an unwavering commitment to reliability, security, and compliance. Concurrence, a company at the forefront of developing clinical AI agents, is demonstrating how to meet these stringent requirements at a significant scale. By consolidating its operational and developer AI workloads onto the Databricks platform, Concurrence is processing an immense volume of data—approximately 100.8 billion input tokens and 11.2 million LLM calls every 30 days, equating to an annualized run rate of roughly 1.2 trillion input tokens. This represents a fivefold increase in monthly token volume since late 2025, underscoring the rapid growth and critical nature of their AI initiatives.

Building Reliable Clinical AI on Trusted Data

Healthcare data is notoriously fragmented and often contains inconsistencies. Concurrence tackles this challenge by treating new information as immutable events rather than overwriting existing records. This approach, which they term their world model, preserves the source and provenance of each data point, allowing AI agents to maintain a consistent and historically accurate view of patient information. This foundation is built upon Databricks Lakehouse Platform, with Zerobus Ingest streaming events into governed Delta tables. Apache Spark™ Declarative Pipelines are used to derive the world model and clinical data, while Unity Catalog governs each customer environment. Lakebase then serves the essential patient state, agent, and conversation state data required by operational applications, enabling workflows like care gap outreach and medication adherence reminders.

Rigorous Testing for Clinical AI Agents

Before any AI agent interacts with a real patient, Concurrence subjects it to extensive testing using simulated patients. The volume of simulation and evaluation traffic on their new platform is currently seven times greater than production traffic. This rigorous testing is made possible by their data architecture, which allows for the replay of patient states derived from immutable event histories without altering actual patient records. Agent traces are ingested into Delta tables alongside clinical data, and scheduled ai_query jobs using Databricks-hosted Claude score interactions for quality and safety. This comprehensive approach ensures that AI agents are thoroughly vetted for various scenarios, enhancing their reliability and safety in clinical settings.

Enforcing AI Governance and Compliance in Healthcare

Compliance with regulations like HIPAA, GDPR, and SOC 2 is a foundational element of Concurrence's architecture. The company is also pursuing HITRUST and ISO 27001/42001 certifications. Unity Catalog plays a crucial role in governing data and AI assets, providing access controls, lineage, and audit trails for each healthcare organization's environment. For batch AI workloads, Concurrence utilizes Databricks-hosted Claude under a Business Associate Agreement (BAA), with an endpoint resolver that strictly prevents Protected Health Information (PHI) from being routed to non-compliant models. This compliance-first approach dictates model routing decisions, prioritizing regulatory adherence over cost or performance.

Governing Developer AI with Unity Gateway

Concurrence extends its governance strategy to developer AI tools, which are used across engineering, operations, and research teams, including those working with sensitive healthcare data. All coding agent model and tool traffic is routed through Unity Gateway’s coding CLI, known as ug. This provides a single, governed pathway to approved models and tools, ensuring that each request is logged and attributed to the user. Unity Gateway centrally manages access to MCP tools, assigning permissions by engineer group and requiring individual authentication. In July, 14 users generated over 35 billion input tokens through Unity Gateway, with a significant portion being cache reads, demonstrating the efficiency of this centralized access point.

Centralizing AI Access for Unified Control

The overarching goal for Concurrence is to unify production, batch, and developer AI under a common inference control point using Unity Gateway. While developer AI is already routed through it, batch inference utilizes Databricks-hosted models via BAA-covered paths. The company employs a multi-model strategy for its clinical workloads, with 14 models currently serving production inference and a governed catalog of 46 models. Smaller, faster models handle most production volumes, while more advanced models are reserved for complex reasoning tasks. Concurrence is actively developing clinical reasoning benchmarks to identify optimal model performance for various healthcare applications and is exploring Unity Gateway Smart Routing to enhance model selection within compliance boundaries.

A Unified Foundation for Scalable Healthcare AI

As Concurrence continues to migrate more workflows onto Databricks, the value of its unified data foundation grows exponentially. Each agent's interaction enriches the patient state, enabling subsequent workflows to leverage existing context, thereby reducing the incremental cost and effort of developing new AI applications. With an annualized run rate of approximately 1.2 trillion production input tokens, this compounding effect is substantial. By integrating patient context, operational state, traces, evaluations, governance, and AI access on a single platform, Concurrence is well-positioned to scale high-stakes clinical AI while upholding the critical reliability and controls demanded by the healthcare industry.

Get the latest posts in your inbox

Subscribe to our blog and get the latest posts delivered to your inbox.

Sign up

View all blogs