ZenML
Blog

Tag: wandb

5 posts with this tag

The Evaluation Playbook: Making LLMs Production-Ready

The Evaluation Playbook: Making LLMs Production-Ready

A comprehensive exploration of real-world lessons in LLM evaluation and quality assurance, examining how industry leaders tackle the challenges of assessing language models in production. Through diverse case studies, the post covers the transition from traditional ML evaluation, establishing clear metrics, combining automated and human evaluation strategies, and implementing continuous improvement cycles to ensure reliable LLM applications at scale.

Dec 14, 20247 mins
Prompt Engineering & Management in Production: Practical Lessons from the LLMOps Database

Prompt Engineering & Management in Production: Practical Lessons from the LLMOps Database

Practical lessons on prompt engineering in production settings, drawn from real LLMOps case studies. It covers key aspects like designing structured prompts (demonstrated by Canva's incident review system), implementing iterative refinement processes (shown by Fiddler's documentation chatbot), optimizing prompts for scale and efficiency (exemplified by Assembled's test generation system), and building robust management infrastructure (as seen in Weights & Biases' versioning setup). Throughout these examples, the focus remains on systematic improvement through testing, human feedback, and error analysis, while balancing performance with operational costs and complexity.

Dec 11, 20247 mins

Popular Topics

+93 more topics