Building LLM Applications with DSPy Review: Beyond Traditional Prompt Engineering

Building LLM applications often begins with a simple prompt. You write an instruction, test the output, adjust a few words, and try again. DSPy takes a different approach through prompt programming: developers define the task and expected inputs and outputs, evaluate performance with metrics, and use optimizers to improve instructions and examples instead of repeatedly rewriting prompts by hand.

Disclosure: This review contains affiliate links. If you purchase through these links, we may earn a small commission at no additional cost to you. Your support helps us continue creating free learning resources on Noro Insight.

In this Building LLM Applications with DSPy Review, I look at a book written by Serj Smorodinsky, a DSPy contributor, data scientist, and AI engineer, and Brett Kennedy, an experienced software developer and data scientist. Published by Manning Publications, the book focuses on prompt programming, evaluation, optimization, and building practical LLM applications with DSPy.

The book progresses from DSPy fundamentals to evaluation, optimization, summarization, classification, RAG, and agentic applications. My review looks at how well it teaches these concepts, how practical the projects are, and who is likely to benefit most from reading it.

Seven Highlights From Building LLM Applications with DSPy

The book moves from DSPy fundamentals into evaluation, optimization, custom modules, summarization, RAG, and agentic applications, using practical projects such as intent classification and dialogue summarization to connect the concepts. The seven points below highlight the parts of the book that stood out most to me.

New to AI? AI Fundamentals: A Beginner’s Guide to Artificial Intelligence

1. Moving From Prompt Engineering to Prompt Programming

One of the book’s most important ideas is the shift from manually rewriting prompts to a more structured form of prompt programming.

Prompt programming means defining what an LLM should do, how its performance will be measured, and letting the system optimize the prompts systematically.

Traditional prompt engineering often means changing instructions, adding examples, testing the output, and repeating the process until the prompt seems good enough. DSPy takes a different approach. Developers define what the task should do, specify the expected inputs and outputs, choose examples and evaluation metrics, and then use DSPy to improve the instructions systematically.

The key shift is from asking, “How should I rewrite this prompt?” to “How should I define, evaluate, and improve this task?” For readers used to manual prompt tuning, this is one of the clearest and most valuable ideas in the book.

Read more: How to Use AI Prompts to Learn Any Skill Faster

2. Signatures and Modules Make DSPy Easier to Understand

DSPy can feel unusual at first, especially if your normal workflow is simply sending a prompt string to an LLM API.

The book makes this easier by gradually introducing DSPy’s main components.

A signature describes the relationship between the inputs of a task and the output you expect. The book shows signatures through examples involving classification, confidence scores, summarization, translation, entailment, and style transfer. Modules then provide reusable components that use those signatures to perform the task. Predictions represent the structured results returned by those modules.

The book also covers practical details such as configuring language models, using LiteLLM, caching, adjusting model parameters, and switching between models.

For me, this is where DSPy begins to feel less like another prompting library and more like a programming framework for LLM applications.

3. Evaluation Is Treated as Part of Development

One of the strongest parts of the book is the attention it gives to evaluation.

It is easy to test a few LLM outputs, see good results, and assume the application is working well. But once the prompt, model, or configuration changes, the real question is:

Did the application actually improve?

The book treats evaluation as part of the development process. It shows how to define metrics, evaluate a baseline, compare results, and measure whether later changes produce meaningful improvement.

Explore NLP: Mastering NLP: A Journey Through Sentiment Analysis

This becomes even more important for open-ended tasks such as summarization, where the book explores approaches including lexical similarity, embeddings, and LLM-as-a-judge.

The main lesson is simple: evaluation should come before optimization, not after it.

4. The Baseline-First Methodology Makes the Book Practical

A recurring idea throughout the book is to establish a baseline before attempting optimization. The authors treat this as a core part of the development process rather than something optional.

The workflow is straightforward:

Without a baseline, it is difficult to know whether changes to prompts, examples, models, or retrieval actually improve the system. The book repeatedly uses this process across its projects so that optimization is measured against a clear starting point.

For readers with a machine learning or data science background, this approach should feel familiar. More importantly, it is a useful discipline even beyond DSPy because it encourages LLM development to be measured, repeatable, and evidence-driven.

5. Prompt Optimization Gets Serious Attention

Optimization is one of the strongest parts of the book. The authors separate it into two areas: improving prompt examples and improving prompt instructions.

For examples, the book covers methods such as LabeledFewShot, BootstrapFewShot, and KNN-based selection. Instruction optimization goes further with tools such as COPRO, MIPROv2, SIMBA, GEPA, and ensemble approaches.

The main idea is to move beyond manually testing different prompts. With DSPy prompt optimization, developers define a metric and let the framework search for better examples or instructions.

The book also keeps the process realistic: optimization still depends on good task definitions, examples, data, and evaluation metrics.

6. Custom Modules Show How DSPy Can Grow With an Application

The book also moves beyond built-in DSPy components and shows how developers can create custom modules for more specific workflows and application needs.

In Building LLM Applications with DSPy, the authors introduce modules such as ChainOfThought, BestOfN, MultiChainComparison, and Refine, then move into custom implementations for tasks such as security checks, critic-based workflows, input transformation, and other specialized behaviors.

The book also covers parallel and asynchronous execution, showing how multiple modules can work together more efficiently. It then returns to optimization and demonstrates how custom modules can also be improved using DSPy optimizers.

This section helps show that DSPy is not limited to simple prompting or a fixed set of predefined components. Developers can combine modules into larger, reusable workflows and adapt them to the requirements of more complex LLM applications.

7. From Summarization to RAG, Agents, and MCP

The later chapters show how DSPy can be applied to more complex LLM systems. The summarization section introduces harder evaluation problems and explores methods such as embeddings and LLM-as-a-judge.

The final chapter brings many of the book’s ideas together through an agentic RAG-based chatbot. It covers baseline RAG, factuality evaluation, multihop RAG, HyDE, ReAct agents, memory, and self-correction.

The book also introduces MCP integration, showing how a DSPy agent can connect to external tools and knowledge through an MCP server.

This final section makes it clear that DSPy is not only about optimizing prompts. It can also support larger LLM workflows that combine retrieval, reasoning, evaluation, memory, and tools.

What I Liked About the Book

The strongest part of Building LLM Applications with DSPy is that it does not treat DSPy as a shortcut for writing better prompts.

The authors build the discussion around a more disciplined development process.

Define the task. Establish a baseline. Evaluate the result. Optimize it. Measure again.

That sequence gives the book a practical foundation.

I also liked the progression from relatively simple problems into more complete applications. Starting with classification makes concepts such as metrics and optimization easier to understand before the book moves into summarization, RAG, and agentic systems.

Another strength is the emphasis on evaluation.

LLM tutorials often spend far more time teaching generation than measurement. This book gives evaluation a central role, which makes sense when optimization is one of DSPy’s main advantages.

The use of open datasets and complete projects also makes it easier for readers to experiment rather than simply copy small isolated code snippets.

What Could Be Better

DSPy has its own learning curve, and concepts such as signatures, modules, metrics, and optimizers may take time to feel natural.

For readers exploring LLM application development with DSPy, the book is deliberately focused on this framework, so broader comparisons with other LLM tools or orchestration platforms will require additional resources.

Because DSPy is still evolving, some APIs and implementation details may change over time. The more lasting value of the book lies in its broader ideas around structured LLM development, evaluation, baselines, modularity, and optimization.

It is also a book that rewards hands-on practice. Running the examples, defining metrics, and comparing optimized results will teach far more than reading the concepts alone.

Who Is This Book For?

The book is best suited to developers, data scientists, AI engineers, and other practitioners who want a more structured way to build LLM applications with DSPy.

Basic Python knowledge is enough, but readers should be comfortable reading and modifying code.

It is especially useful for people interested in prompt programming, evaluation, optimization, RAG, and agentic workflows rather than the internal theory of Transformers or LLM training.

Is Building LLM Applications with DSPy Worth Reading?

Yes, especially for readers who want a structured and approachable introduction to DSPy. The book is easy to follow, builds concepts gradually, and is designed for readers with only basic Python skills.

Its strongest value is how it connects prompt programming, evaluation, baselines, and optimization into one practical workflow rather than teaching DSPy as a collection of isolated features.

For beginners to DSPy, data scientists, and developers moving beyond manual prompt engineering, it offers a friendly starting point without requiring advanced programming knowledge.

Final Thoughts

Building LLM Applications with DSPy is more than a guide to writing better prompts. Its real strength is showing how LLM applications can be treated as systems that are defined, evaluated, tested, and improved in a more systematic way.

The book makes DSPy easier to understand by connecting prompt programming with baselines, metrics, optimization, and practical projects such as classification, summarization, RAG, and agentic applications. That progression helps readers see where DSPy fits beyond traditional prompt engineering.

For developers, data scientists, and AI practitioners who want a structured and beginner-friendly path into DSPy, this book offers a strong combination of concepts and hands-on application.

Overall, it is a valuable resource for learning how to move from manual prompt tuning toward more measurable and reliable LLM development.