TCS and MongoDB present a case study on modernizing data infrastructure by integrating Operational Data Layers (ODLs) with generative AI and vector search capabilities. The solution addresses challenges of fragmented, outdated systems by creating a real-time, unified data platform that enables AI-powered insights, improved customer experiences, and streamlined operations. The implementation includes both lambda and kappa architectures for handling batch and real-time processing, with MongoDB serving as the flexible operational layer.
This article, authored by Ray Hassan and Bikram Sankar Das from TCS (Tata Consultancy Services) and published on the MongoDB blog, presents a conceptual framework for building unified data platforms that support generative AI workloads. It is important to note upfront that this is primarily a marketing and thought-leadership piece rather than a detailed production case study with specific metrics and outcomes. The content describes architectural patterns and theoretical benefits rather than documenting a specific customer implementation with measurable results.
The core premise is that organizations relying on legacy systems—particularly 1970s-era mainframes still common in industries like banking—face significant challenges in adopting AI due to fragmented, siloed data that is often stale by the time it reaches decision-makers. The proposed solution centers on implementing Operational Data Layers (ODLs) powered by MongoDB, combined with generative AI and vector search capabilities.
The ODL serves as a centralized hub that integrates data from multiple transactional systems in real-time. This architectural pattern is positioned as the foundation for enabling generative AI in production environments. The ODL acts as an intermediary between data producers and consumers, consolidating information that would otherwise be trapped in silos across CRM, HR, billing, and other enterprise systems.
From an LLMOps perspective, the ODL addresses a fundamental challenge: LLMs require access to current, accurate, and contextually relevant data to provide useful outputs. Without a unified data layer, organizations attempting to deploy LLMs face the choice of either working with stale batch-processed data or building complex point-to-point integrations between AI systems and multiple data sources.
The document outlines several use cases for ODL-enabled Gen AI:
The article includes a conceptual diagram showing Retrieval Augmented Generation (RAG) implementation using what they term a “converged AI data store.” In this architecture, the data store provides context to LLM prompts, addressing the common challenge of grounding LLM outputs in organizational knowledge.
The RAG pattern is particularly relevant for LLMOps because it allows organizations to enhance LLM responses with proprietary data without requiring model fine-tuning. The MongoDB platform is positioned as suitable for this purpose because it can store both the operational data and the vector embeddings needed for semantic search in a single database.
Vector search is highlighted as a key enabling technology. The article describes vector search as translating high-dimensional data like text, images, and audio into vectors, enabling “accurate, intuitive searches.” An example given is healthcare providers searching for similar patient cases by symptoms to enhance diagnostics—though this remains at the conceptual level without specific implementation details.
The document discusses two primary architectural patterns for organizations modernizing their data platforms to support AI workloads:
Lambda Architecture: This approach processes both batch and real-time data through three layers:
Kappa Architecture: For organizations focused primarily on real-time analytics, this simpler architecture uses a single stream for data processing. MongoDB is positioned as the operational speed layer, supporting high-speed, real-time data updates enhanced by Gen AI.
The choice between these architectures has significant implications for LLMOps deployments. Lambda architecture may be more suitable when LLMs need access to both historical context (for training or fine-tuning purposes) and real-time operational data. Kappa architecture may be preferable for applications where LLMs primarily need current state information rather than historical analysis.
The article outlines a progressive journey toward data modernization that enables AI capabilities:
This phased approach is relevant to LLMOps practitioners because it acknowledges that AI capabilities cannot simply be bolted onto existing infrastructure. The underlying data platform must be modernized to support the real-time, flexible access patterns that LLM applications require.
The article includes additional content about telecommunications industry applications, providing somewhat more concrete examples of ODL and Gen AI integration:
These examples, while still largely conceptual, suggest production-scale deployments in the telecommunications sector.
It is important to approach this content with appropriate skepticism. The article is fundamentally a marketing piece published by MongoDB, co-authored with a consulting partner (TCS). Key observations:
That said, the underlying architectural principles are sound. The integration of operational data layers with vector search capabilities for RAG implementations represents a legitimate approach to deploying LLMs in enterprise environments. The emphasis on real-time data access and the progression from batch to streaming architectures reflects genuine requirements for production AI systems.
The article serves as a useful introduction to thinking about data platform requirements for generative AI but should be supplemented with more detailed implementation guidance and independent case studies for practitioners planning actual deployments.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
New Relic, a major observability platform processing 7 petabytes of data daily, implemented GenAI both internally for developer productivity and externally in their product offerings. They achieved a 15% increase in developer productivity through targeted GenAI implementations, while also developing sophisticated AI monitoring capabilities and natural language interfaces for their customers. Their approach balanced cost, accuracy, and performance through a mix of RAG, multi-model routing, and classical ML techniques.
John Snow Labs developed a comprehensive healthcare LLM system that integrates multimodal medical data (structured, unstructured, FHIR, and images) into unified patient journeys. The system enables natural language querying across millions of patient records while maintaining data privacy and security. It uses specialized healthcare LLMs for information extraction, reasoning, and query understanding, deployed on-premises via Kubernetes. The solution significantly improves clinical decision support accuracy and enables broader access to patient data analytics while outperforming GPT-4 in medical tasks.
