LLM Observability with Langfuse
An Independent Guide to Tracing, Evaluating, Prompt Management and Cost Monitoring for AI Applications and Agents
-
- $0.99
-
- $0.99
Publisher Description
A practical, independent guide to observability for applications and agents built on large language models, using the open-source Langfuse platform. Following one running project, a tutoring assistant for a fictional online education company, the book explains why language model systems need a different kind of monitoring and then shows how to capture, read and act on traces. It covers the data model of traces, observations, sessions and scores; OpenTelemetry and the GenAI semantic conventions; instrumentation with the OpenTelemetry-based Python SDK; integrations with providers and agent frameworks; debugging agents, tools and retrieval; and prompt management. It then covers scores, user feedback, annotation queues, LLM-as-a-judge and code evaluators, datasets, experiments and quality gates in continuous integration. The final part covers cost tracking, dashboards and alerts, masking and retention under several privacy laws, self-hosting with PostgreSQL, ClickHouse, Redis and object storage on Docker and Kubernetes, scaling, upgrades and a fair comparison with alternatives. Includes checklists, a hands-on lab, study questions with answers and a glossary.
This book is an independent publication and is not authorized, sponsored or endorsed by any company whose products it mentions. All trademarks belong to their respective owners and are used only for identification. This book was drafted with generative AI tools and checked with automated tools. Tools, features and prices change often; confirm them in current documentation.