Altana, a global supply chain intelligence company, faced challenges in efficiently deploying and managing multiple GenAI models for diverse customer use cases. By implementing Databricks Mosaic AI platform, they transformed their ML lifecycle management, combining custom deep learning models with fine-tuned LLMs and RAG workflows. This led to 20x faster model deployment times and 20-50% performance improvements, while maintaining data privacy and governance requirements across their global operations.
Altana is a growth-stage company building what they describe as the world’s first value chain management system, focused on making global supply chains more transparent and reliable. The company operates across multiple countries on three continents, serving over a thousand customers with approximately 200 employees. Their core business is deeply rooted in data and machine learning, which forms the foundation of their product offerings for global trade challenges.
The case study presents Altana’s journey from struggling with infrastructure overhead and workflow inefficiencies to achieving significant improvements in model deployment speed and performance through the adoption of Databricks’ platform. While the claims of 20x faster deployment and 20-50% better performance are impressive, these are self-reported metrics from a customer story published by the platform vendor, so they should be viewed with appropriate context.
Before adopting Databricks, Altana faced several operational challenges that are common in LLMOps at scale:
Altana employs generative AI for several production use cases that demonstrate practical applications of LLMs in the supply chain domain:
Tax and Tariff Classification: One of their primary use cases involves streamlining cross-border trading by generating accurate tax and tariff classifications for goods. The system not only classifies goods but also auto-generates comprehensive legal write-ups that customers can use for justification purposes. This is a compelling example of using LLMs to reduce manual work in a domain that traditionally requires significant expertise.
Supply Chain Knowledge Graph Enhancement: Another use case focuses on enriching their global supply chain knowledge graph by integrating customer data in a privacy-preserving manner. This enables better understanding of supply chain dependencies and supports strategic decision-making. The privacy-preserving aspect is notable as it addresses a key concern in enterprise AI deployments.
Altana’s implementation demonstrates a sophisticated compound AI systems approach that combines multiple techniques:
Multi-Stage Workflow Design: Their most advanced workflows use custom deep learning models for early processing phases, followed by fine-tuned agent workflows with retrieval augmented generation (RAG) for refinements. This layered approach allows them to leverage the strengths of different model types at different stages of the pipeline.
RLHF Integration: Reinforcement learning from human feedback (RLHF) from actual customers is used for continuous model refinement. This creates a feedback loop where production usage directly improves model quality over time. The platform is used to track all user input/output decisions, enabling identification of which steps and models in the workflow need fine-tuning for cost, performance, or latency optimization.
Model Selection Strategy: The team takes a pragmatic approach to model selection, combining existing open-source LLMs like DBRX, Llama 3, and Phi-3 with their internal proprietary datasets for RAG workflows. These open-source models also serve as backbones for some of their deep learning solutions. This hybrid approach balances the capabilities of state-of-the-art open models with domain-specific customization.
Managed MLflow: This tool was described as pivotal in managing Altana’s extensive ML lifecycle. The model and LLM evaluation tools in MLflow were specifically highlighted as enabling ML teams to rapidly train, test, and deploy their own models for customers. The evaluation capabilities are particularly important for production LLM deployments where traditional metrics may not capture quality effectively.
Model Serving: Combined with MLflow’s evaluation tools, Model Serving provides the deployment infrastructure needed to move models from development to production. The “batteries included” nature of having these tools integrated in one platform was cited as enabling better collaboration across data ingest, data engineering, ML, and software teams.
Delta Lake: The team migrated all their data to the Delta format approximately a year before the case study was published. Several features were highlighted as beneficial:
Unity Catalog: For a company dealing with sensitive supply chain and customer data, governance is critical. Unity Catalog provides data governance tools ensuring customer data privacy and security. The case study emphasizes their federated architecture and privacy-preserving approach where all customer data is kept separate and private—a requirement for enterprise customers in regulated industries.
An interesting aspect of Altana’s implementation is their early adoption of agentic workflows. They mention already integrating agentic workflows into their ETL process for enrichment, validation, and normalization tasks. This represents an emerging pattern in LLMOps where LLMs are not just serving end-user requests but are embedded into data pipelines to improve data quality and processing.
The company plans to expand this usage further, suggesting they see significant value in AI-augmented data engineering. This aligns with broader industry trends toward using LLMs for data transformation and quality tasks.
The case study claims the following improvements:
It’s worth noting that these metrics come from a vendor-published customer story, so they should be interpreted with appropriate skepticism. However, the qualitative benefits described—reduced infrastructure overhead, better team collaboration, and integrated tooling—are consistent with well-documented benefits of unified ML platforms.
While the case study presents compelling results, several aspects warrant consideration:
The performance metrics (20x faster, 20-50% better) are dramatic but lack detailed methodology. What baseline are these improvements measured against? How was model performance quantified, especially for generative outputs like legal write-ups?
The case study is published by Databricks, so it naturally emphasizes positive outcomes. Real-world implementations typically involve more nuanced tradeoffs and challenges that may not be fully represented.
The emphasis on avoiding vendor lock-in through Databricks’ “open architecture” is somewhat ironic given that adopting a comprehensive platform like Databricks creates its own form of dependency. However, the mention of being able to “swap in specific models, regional deployments, or optimize specific components as needed” suggests the architecture does provide flexibility.
This case study illustrates several important patterns for production LLM deployments:
The Altana case represents a mature LLMOps implementation that goes beyond simple LLM API calls to build sophisticated production systems combining multiple AI techniques for real business value in the supply chain domain.
New Relic, a major observability platform processing 7 petabytes of data daily, implemented GenAI both internally for developer productivity and externally in their product offerings. They achieved a 15% increase in developer productivity through targeted GenAI implementations, while also developing sophisticated AI monitoring capabilities and natural language interfaces for their customers. Their approach balanced cost, accuracy, and performance through a mix of RAG, multi-model routing, and classical ML techniques.
Wesco, a B2B supply chain and industrial distribution company, presents a comprehensive case study on deploying enterprise-grade AI applications at scale, moving from POC to production. The company faced challenges in transitioning from traditional predictive analytics to cognitive intelligence using generative AI and agentic systems. Their solution involved building a composable AI platform with proper governance, MLOps/LLMOps pipelines, and multi-agent architectures for use cases ranging from document processing and knowledge retrieval to fraud detection and inventory management. Results include deployment of 50+ use cases, significant improvements in employee productivity through "everyday AI" applications, and quantifiable ROI through transformational AI initiatives in supply chain optimization, with emphasis on proper observability, compliance, and change management to drive adoption.
A panel discussion featuring leaders from Nubank, Harvey AI, Galileo, and Convirza discussing their experiences implementing LLMs in production. The discussion covered key challenges and solutions around model evaluation, cost optimization, latency requirements, and the transition from large proprietary models to smaller fine-tuned models. Participants shared insights on modularizing LLM applications, implementing human feedback loops, and balancing the tradeoffs between model size, cost, and performance in production environments.
