Building LLM Applications with DSPy (Replacing manual prompts with systematic optimization)

“An essential, forward-looking guide.”
—Dhyey Mavani, Amherst College

DSPy (Declarative Self-improving Python), an innovative framework for prompt programming, replaces fragile and unpredictable manual prompts with clean, modular Python code that declares what a model should do, defines the inputs and response formats, and establishes clear evaluation metrics. In Building LLM Applications with DSPy, authors Serj Smorodinsky and Brett Kennedy present a systematic approach for treating prompts as programmatic artifacts rather than loose text strings, so you can build, scale, maintain, and improve complex AI applications with the structural integrity of traditional software.

Practical from page one, this book mirrors a professional AI engineering workflow, taking you from simple classifiers to complex summarizers, advanced RAG, and agentic systems. Reviewer Cyrus Nouroozi, a DSPy contributor, notes that the book “frames the whole subject around a single coherent thesis: prompt programming is a data-driven discipline analogous to machine learning.” To support its strong focus on prompt programming as an engineering process, the book introduces the baseline —> evaluate —> optimize loop that eliminates the guesswork of traditional prompt design.

Production applications require verifiable, consistent metrics, which are difficult to establish with conventional prompts. This book guides you through constructing custom evaluation metrics, setting up multi-threaded test runners, and calibrating LLM-as-a-judge protocols to gather hard data. Instead of deploying prompts on a whim, you compile and optimize them against structured validation sets. The resulting programs are mathematically proven to be more accurate, more cost-effective, and remarkably resilient to underlying model drift.

The final chapters show you how to build DSPy into fully agentic pipelines, integrating conversation memories, and connecting agents to external environments via MCP. This focused 9-chapter book also introduces cutting-edge DSPy v3 features like SIMBA and GEPA optimizers to squeeze maximum performance out of smaller, cheaper open-weight language models. It is an indispensable resource for any modern programmer wanting to build robust, self-improving, and production-ready generative AI systems.

What's inside

• Build modular LLM applications using declarative Python signatures
• Automate prompt optimization using SIMBA and GEPA
• Construct rigorous, repeatable metric functions to verify outputs
• Deploy reliable, multi-hop RAG systems and agents

About the reader

This book is designed for software developers, data scientists, and AI engineers with basic Python skills.

About the author

Serj Smorodinsky is a DSPy contributor, data scientist, and AI engineer with over ten years of experience leading teams to build conversational AI, conversational chatbots, and agentic workflows for enterprise clients. Brett Kennedy is a data scientist with over thirty years of software development experience. He is a regular contributor to open-source projects and the author of Outlier Detection in Python.

Table of Contents

1 Introduction to prompt programming and DSPy
2 Basic prompting and DSPy
3 Classifying user intent
4 Evaluating DSPy programs
5 Optimizing prompt examples
6 Optimizing prompt instructions
7 Custom modules
8 Summarization and more effective metric functions
9 Creating an agentic RAG-based chatbot