Patho AI developed a Knowledge Augmented Generation (KAG) system for enterprise clients that goes beyond traditional RAG by integrating structured knowledge graphs to provide strategic advisory and research capabilities. The system addresses the limitations of vector-based RAG systems in handling complex numerical reasoning and multi-hop queries by implementing a "wisdom graph" architecture that captures expert decision-making processes. Using Node-RED for orchestration and Neo4j for graph storage, the system achieved 91% accuracy in structured data extraction and successfully automated competitive analysis tasks that previously required dedicated marketing departments.
Patho AI, founded by CEO Ching Kyong Lamb, began as an NSF SBIR grant-funded company two years ago, initially focused on LLM-driven drug discovery applications. The company has since pivoted to building expert AI systems for large corporate clients, moving beyond traditional RAG (Retrieval Augmented Generation) systems to develop what they term “Knowledge Augmented Generation” (KAG) systems that perform research and advisory roles based on clients’ specific domains of expertise.
The fundamental challenge that Patho AI identified was that many enterprise clients were requesting AI systems capable of sophisticated reasoning and advisory functions that traditional RAG systems couldn’t adequately address. While RAG systems excel at retrieving relevant documents based on semantic similarity, they struggle with complex numerical calculations, multi-hop reasoning, and the type of strategic thinking that human experts employ in their decision-making processes.
The company’s primary innovation centers on transitioning from traditional vector-based RAG to Knowledge Augmented Generation (KAG), which integrates structured knowledge graphs to provide more accurate and insightful responses. The key philosophical difference is that while RAG systems primarily retrieve information, KAG systems are designed to understand and reason over interconnected knowledge structures.
The foundation of their approach rests on a conceptual framework they call the “wisdom graph” - a systematic method of preserving and connecting domain expertise through interconnected relationships. This graph represents not just static information but the dynamic thought processes and comprehensive taxonomy of specific domains of expertise. The wisdom graph architecture consists of several interconnected components: wisdom (the core reasoning engine), knowledge (facts and data), experience (historical patterns and outcomes), insights (derived patterns and strategic recommendations), and situational awareness (current context and conditions).
One of the more interesting technical decisions made by Patho AI was their choice to use Node-RED, typically known as a workflow automation tool for IoT applications, as their orchestration platform for multi-agent LLM systems. While this might seem unconventional, the CEO draws parallels to IBM’s approach in IoT projects where Node-RED served as a no-code interface overlaying more complex underlying systems.
The Node-RED implementation allows for flexible prototyping and rapid development of complex state machine workflows. The system includes specialized AI agent nodes that can interface with different LLM providers including OpenAI, Anthropic, and on-premises models. This multi-model approach provides both flexibility and redundancy, allowing clients to choose models based on their specific requirements for accuracy, cost, or data privacy.
The system utilizes Neo4j as the graph database backend, leveraging Cypher queries for complex graph traversals. The knowledge graph structure goes beyond simple entity-relationship models to capture the nuanced decision-making patterns that human experts use. The graph is continuously updated by different specialized agents, each responsible for maintaining specific aspects of the domain knowledge.
The extraction process for populating the knowledge graph uses a hybrid approach that combines automated LLM-based extraction with human expert validation. This hybrid model addresses one of the key challenges in knowledge graph construction - ensuring both coverage and accuracy. The automated LLM graph transformer handles the initial extraction, while domain experts prune the graph by removing irrelevant relationships and validating the taxonomy structure.
A specific client implementation involved building a competitive intelligence system for a company that previously relied on their marketing department for competitive analysis. The client wanted to automate this process using AI while maintaining the strategic depth and insight that human analysts provided. Traditional RAG systems proved inadequate for answering sophisticated strategic questions like “How can I win against my competitor in this market space?” because such questions require multi-faceted analysis across various data sources and the ability to perform complex reasoning over numerical data.
The competitive intelligence system maps the abstract wisdom graph concept to concrete business functions:
The system employs multiple specialized agents working in coordination:
All agents contribute to and draw from a centralized knowledge graph that serves as the system’s memory and reasoning foundation. This centralized approach ensures consistency while allowing for specialized processing by domain-specific agents.
The presentation highlighted several key limitations of traditional vector-based RAG systems that their KAG approach addresses:
Numerical Reasoning Deficiency: Vector stores excel at semantic similarity matching but struggle with complex numerical calculations. For marketing analysis requiring precise quantitative insights, traditional RAG might return relevant passages containing numbers but cannot perform the calculations needed to derive actionable metrics. The KAG system addresses this by storing numerical data in structured graph format and enabling function calls for mathematical operations.
Multi-hop Query Complexity: Traditional RAG systems often fail at questions requiring multiple logical steps or relationships. The graph-based approach naturally supports traversing multiple relationships to answer complex, multi-faceted queries that are common in strategic business analysis.
Context Preservation: Vector similarity can lose important contextual relationships between entities. Knowledge graphs maintain these relationships explicitly, enabling more nuanced understanding of how different business factors interact.
The system is designed to work alongside traditional RAG when appropriate, creating a hybrid architecture that can adapt to different types of queries:
Patho AI conducted internal benchmarking that showed significant improvements over traditional RAG approaches across several metrics:
While these metrics appear impressive, it’s important to note that the specific benchmarking methodology and comparison baselines were not detailed in the presentation, which is common in vendor presentations but limits the ability to independently verify these claims.
The practical impact for the competitive intelligence client included automation of previously manual processes, more consistent analysis across time periods, and the ability to handle more complex strategic questions than traditional business intelligence tools. However, the presentation did not provide specific quantitative measures of business impact such as time savings, cost reduction, or improved decision outcomes.
The system architecture demonstrates thoughtful consideration of production deployment needs. The use of Node-RED provides a visual development environment that can potentially be maintained by non-technical stakeholders, while the underlying complexity is handled by more sophisticated components. The multi-model support (OpenAI, Anthropic, on-premises) allows organizations to balance performance, cost, and data privacy requirements.
The hybrid knowledge extraction approach acknowledges the ongoing maintenance requirements of knowledge graph systems. By combining automated extraction with human expert validation, the system aims to balance automation benefits with accuracy requirements. This approach recognizes that domain expertise remains crucial for ensuring the quality and relevance of the knowledge representation.
Patho AI has made some components available as open source, including what they describe as an “LLM graph RAG stack” that can be deployed via Docker. This approach to sharing core infrastructure while maintaining proprietary expertise in application and customization reflects a common pattern in the enterprise AI space.
It’s important to note that this case study is based on a conference presentation by the company’s founder, which naturally focuses on successes and achievements rather than challenges or limitations. The benchmarking results, while impressive, lack detailed methodology descriptions that would allow for independent verification. The specific business impact metrics (ROI, time savings, error reduction) that would typically validate enterprise AI deployments are not provided.
While the KAG approach addresses genuine limitations of traditional RAG systems, it introduces significant complexity in terms of knowledge graph design, maintenance, and updating. The success of such systems heavily depends on the quality of the initial knowledge modeling and ongoing curation, which can be resource-intensive. The hybrid approach requiring domain expert involvement for graph pruning may limit the scalability benefits that organizations expect from AI automation.
The competitive intelligence use case represents a specific type of analytical problem well-suited to knowledge graph approaches. The generalizability of this architecture to other enterprise use cases, particularly those with less structured decision-making processes or more diverse data types, remains an open question that would need to be validated through broader deployment.
Despite these limitations, Patho AI’s approach represents meaningful innovation in addressing real limitations of current RAG systems. The integration of structured reasoning with LLM capabilities, the thoughtful use of multi-agent orchestration, and the hybrid approach to knowledge extraction all represent practical advances in enterprise AI system design.
The work also demonstrates the evolution of LLMOps beyond simple question-answering systems toward more sophisticated expert system capabilities. This trend toward AI systems that can perform advisory and strategic functions represents a significant step toward more valuable enterprise AI applications.
The choice to build on established tools like Node-RED and Neo4j, while potentially limiting in some ways, also demonstrates a pragmatic approach to enterprise deployment that prioritizes reliability and maintainability over cutting-edge technical innovation. This balance between innovation and practicality is often crucial for successful enterprise AI deployments.
In the broader context of LLMOps evolution, this case study illustrates the growing sophistication of production LLM systems and the need for more complex orchestration, knowledge management, and reasoning capabilities as organizations move beyond basic chatbot applications toward systems that can truly augment human expertise and decision-making capabilities.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
Stripe, processing approximately 1.3% of global GDP, has evolved from traditional ML-based fraud detection to deploying transformer-based foundation models for payments that process every transaction in under 100ms. The company built a domain-specific foundation model treating charges as tokens and behavior sequences as context windows, ingesting tens of billions of transactions to power fraud detection, improving card-testing detection from 59% to 97% accuracy for large merchants. Stripe also launched the Agentic Commerce Protocol (ACP) jointly with OpenAI to standardize how agents discover and purchase from merchant catalogs, complemented by internal AI adoption reaching 8,500 employees daily using LLM tools, with 65-70% of engineers using AI coding assistants and achieving significant productivity gains like reducing payment method integrations from 2 months to 2 weeks.
This panel discussion features three AI-native companies—Delphi (personal AI profiles), Seam AI (sales/marketing automation agents), and APIsec (API security testing)—discussing their journeys building production LLM systems over three years. The companies address infrastructure evolution from single-shot prompting to fully agentic systems, the shift toward serverless and scalable architectures, managing costs at scale (including burning through a trillion OpenAI tokens), balancing deterministic workflows with model autonomy, and measuring ROI through outcome-based metrics rather than traditional productivity gains. Key technical themes include moving away from opinionated architectures to let models reason autonomously, implementing state machines for high-confidence decisions, using tools like Pydantic AI and Logfire for instrumentation, and leveraging Pinecone for vector search at scale.
