A detailed discussion between Patrick Barker (CTO of Guaros) and Farud (ML Engineer from Iran) about the relevance and future of LLMOps, with Patrick arguing that LLMOps represents a distinct field from traditional MLOps due to different user profiles and tooling needs, while Farud contends that LLMOps may be overhyped and should be viewed as an extension of existing MLOps practices rather than a separate discipline.
This case study captures a spirited debate from an MLOps Community podcast featuring two practitioners with distinctly different perspectives on whether “LLMOps” constitutes a genuinely new discipline or represents marketing hype built atop existing MLOps foundations. The discussion is particularly valuable for understanding the evolving landscape of production ML systems and how the introduction of large language models has potentially shifted the required skillsets, tooling, and operational paradigms.
The two main voices in this debate are:
Patrick makes a strong case that his experience transitioning from traditional MLOps at One Medical to building LLM-based applications revealed almost no overlap in required skills. At One Medical, his daily work involved setting up the standard ML 1.0 tool stack on Kubernetes, using MLflow for experiment tracking, training early transformer models for text classification using SageMaker training, and serving models on platforms like Seldon or KServe. He was building multiclass classification models for detecting things in text—classic NLP work.
When he moved to working with LLMs and building agent-based applications, Patrick found that “there’s hardly any crossover.” Even fine-tuning LLMs is different, he argues, because of the scale involved and the specific techniques like LoRA (Low-Rank Adaptation) that require entirely different tooling than what was used with earlier transformers.
Patrick’s most compelling argument centers on the user persona shift. The “biggest use case with LLMs,” he notes, is “JavaScript developer with an OpenAI key”—developers who were previously outside the ML ecosystem entirely. These frontend and application developers now need tools specifically designed for their workflows, not repurposed MLOps infrastructure. He points to Vercel AI as a perfect example of tooling built specifically for TypeScript developers working with LLMs.
Farood takes a more skeptical stance, viewing LLMOps as part of a pattern of tech industry hype cycles. He draws parallels to the cloud computing bubble around 2012 and expresses concern about “over promising and under delivering” that hinders actual technical practitioners working toward realistic goals.
His core argument is that MLOps represents a mindset and paradigm rather than a specific technology stack. The fundamental problems MLOps tries to solve—delivering data transparently, securely, and reproducibly—don’t change whether you’re training an LLM or a simple logistic regression model. “This NeverEnding cycle from data to production” remains constant regardless of the underlying model architecture.
Farood is particularly concerned about the fragmentation of the field into “very small specific surgical tools” that may not scale well as a discipline. He advocates for extending current tools and tooling rather than creating entirely separate ecosystems for each new paradigm.
Despite their disagreements, both practitioners find common ground on several important points:
Data Engineering Remains Central: Both agree that the data pipeline and data engineering aspects of MLOps and LLMOps are “almost identical.” The problems of data quality, reproducibility, and governance don’t disappear just because you’re working with foundation models. Patrick acknowledges that data engineering is “probably the hardest part to automate” in any ML workflow.
The MLOps Term Itself is Immature: Both note that even the term “MLOps” hasn’t fully established itself. DevOps practitioners are often dismissive of the term, and what MLOps means at Microsoft, Google, or Facebook doesn’t map to what a medium-sized company needs. This existing ambiguity makes the addition of yet another term (“LLMOps”) even more problematic.
Bubbles Have Value: Interestingly, Patrick embraces the idea of hype and bubbles as productive forces that encourage exploration. He cites economic research suggesting that bubble economies actually perform better than gradual growth because they enable exploration of many directions before finding what works. Farood doesn’t necessarily disagree but expresses concern about the impact on working practitioners.
Patrick explicitly addresses the question of whether understanding MLOps provides a “power-up” when working with LLMs. His answer is nuanced: if you’re fine-tuning with LoRA, your MLOps background will be “incredibly beneficial” for understanding base concepts. But if you’re a JavaScript developer building applications on top of API calls, that deep ML knowledge is “a lot lesser” in its utility.
This raises important questions about team composition and hiring for LLMOps roles. The traditional path of DevOps → MLOps → LLMOps may not be the most efficient for all use cases.
Patrick makes a bold prediction: “Gen AI will eat ML 1.0 in less than three years”—possibly even sooner. His reasoning centers on agents becoming increasingly capable of using tools, citing research like “Agent Tuning” that shows models can approach GPT-4 capacity for tool use when specifically trained for it.
He envisions a future similar to how the brain works, with a “higher level brain” (LLMs) that can construct and orchestrate “lower level algorithms” optimized for specific tasks like processing audio, visual, or tabular data. XGBoost isn’t going to be replaced by LLMs directly because it’s “so efficient and so small and so effective at what it does,” but agents could potentially train XGBoost models on tabular data with increasing autonomy.
Companies like Abacus AI are already approaching this “generative AI for MLOps” space, though Patrick acknowledges uncertainty about how well it currently works.
Farood raises an important concern that often gets overlooked in LLMOps discussions: the carbon footprint and energy consumption of training and running large models. He notes that when OpenAI trains a new GPT model, they’re essentially training from scratch—a brute-force approach that may not be sustainable as the field scales.
Patrick acknowledges this as “a huge problem” and notes that there’s a company out of MIT focused specifically on environmental efficiency for LLMs. He also points to the Mamba paper as a potentially promising development, offering “wildly more efficient” computation than transformers with linear context scaling.
A recurring theme in the discussion is that “it’s really really hard to get an AI application on LLMs out to production.” Patrick notes that when he started working with LLM applications, there weren’t really any tools to help with this challenge. This gap has spurred the development of new LLMOps tooling, though there’s significant noise alongside genuinely useful tools.
The discussion touches on the need for evaluation tooling specifically designed for LLM outputs, which differs substantially from traditional ML model evaluation. The stochastic nature of LLM outputs, the importance of context engineering (over prompt engineering, which Patrick sees as a potentially temporary artifact), and the challenges of testing generative systems all require new approaches.
The debate surfaces several practical considerations for those working in or entering the field:
Role Definition: MLOps is already a role that Farood describes as requiring “five different engineers”—data engineering, data science, DevOps, backend development, and BI knowledge. Adding LLM-specific skills to this already broad requirement set creates questions about specialization versus generalization.
Tool Selection: The discussion highlights a tension between building specialized tools for specific use cases (the approach many LLMOps startups are taking) versus extending existing MLOps tools with LLM capabilities (which MLflow and others are doing). Neither approach has clearly won.
Foundation Model Dependency: Many current LLMOps use cases are essentially thin wrappers around API calls to models like GPT-4. This creates questions about long-term defensibility and value creation that wouldn’t arise with traditional ML where you’re training and owning your models.
The Persona Shift: The emergence of frontend and application developers as primary LLM users represents a significant market expansion. Tooling and education need to adapt to serve this new audience, which may mean creating genuinely new tools rather than adapting existing MLOps infrastructure.
It’s worth noting that this discussion, while insightful, represents opinions from two practitioners at a specific moment in time. The field is evolving rapidly, and claims made—particularly Patrick’s prediction about agents eating ML 1.0—should be viewed skeptically. As noted in the discussion, many research papers don’t hold up when reproduced at industry scale, and VC funding doesn’t guarantee technical progress.
The debate also reflects the speakers’ specific contexts: Patrick is building an agent startup and has clear incentives to view LLMOps as a distinct, important field; Farood is implementing MLOps at a more traditional company and sees continuity with existing practices. Both perspectives have validity, but neither represents a complete picture.
What emerges most clearly from this discussion is that the LLMOps space is genuinely in flux, with legitimate debates about fundamentals that won’t be resolved by theoretical argument alone but by practical experience deploying LLM systems at scale across diverse use cases.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
Amazon teams faced challenges in deploying high-stakes LLM applications across healthcare, engineering, and e-commerce domains where basic prompt engineering and RAG approaches proved insufficient. Through systematic application of advanced fine-tuning techniques including Supervised Fine-Tuning (SFT), Proximal Policy Optimization (PPO), Direct Preference Optimization (DPO), and cutting-edge reasoning optimizations like Group-based Reinforcement Learning from Policy Optimization (GRPO) and Direct Advantage Policy Optimization (DAPO), three Amazon business units achieved production-grade results: Amazon Pharmacy reduced dangerous medication errors by 33%, Amazon Global Engineering Services achieved 80% human effort reduction in inspection reviews, and Amazon A+ Content improved quality assessment accuracy from 77% to 96%. These outcomes demonstrate that approximately one in four high-stakes enterprise applications require advanced fine-tuning beyond standard techniques to achieve necessary performance levels in production environments.
Cosine, a company building enterprise coding agents, faced the challenge of deploying high-performance AI systems in highly constrained environments including on-premise and air-gapped deployments where large frontier models were not viable. They developed a multi-agent architecture using specialized orchestrator and worker models, leveraging model distillation, supervised fine-tuning, preference optimization, and reinforcement fine-tuning to create smaller models that could match or exceed the performance of much larger models. The result was a 31% performance increase on the SWE-bench Freelancer benchmark, 3X latency improvement, 60% reduction in GPU footprint, and 20% fewer errors in generated code, all while operating on as few as 4 H100 GPUs and maintaining full deployment flexibility across cloud, VPC, and on-premise environments.
