TL;DR
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
The Pragmatic Engineer published excerpts from Sam Newman’s 2026 book, Building Resilient Distributed Systems, covering disaster recovery and resilience for AI-powered software. The material emphasizes setting recovery targets, weighing resilience against cost and complexity, and preparing for model, vendor and infrastructure changes.
The Pragmatic Engineer has published excerpts from Sam Newman’s 2026 book, Building Resilient Distributed Systems, outlining how software teams can plan for major outages and reliability risks in AI-powered systems. The material covers disaster recovery targets and strategies, as well as the challenges of relying on models and infrastructure that may change or become unavailable.
In the excerpt from Chapter 10, Newman describes disaster recovery as the set of procedures used when failures affect large parts of a system, such as a site-wide outage caused by fire, flooding, power problems or an issue in cloud infrastructure. He says teams should define a recovery point objective (RPO) and a recovery time objective (RTO) before selecting a recovery approach. RPO sets the maximum acceptable period of potential data loss; RTO sets how long a system can remain unavailable.
The report lists several disaster-recovery patterns: backup and restore, pilot-light failover, warm failover and multisite active-active operation. These options involve different costs, levels of complexity and recovery capabilities. Newman’s excerpt says an RTO must account not just for making computing resources available, but also restoring data and redirecting traffic to another site.
The newsletter’s summary says other excerpts address the resilience of AI-powered software, including model-provider reliability, models changing over time and the need for observability. It also points to uncertainty-management tactics such as using more than one model or vendor, making costs transparent and reviewing vendors regularly. The report flags ongoing GPU and CPU shortages as a reason not to assume replacement infrastructure will always be available when needed.
Recovery Targets Shape Outage Plans
The guidance matters because a recovery plan is a set of operational choices, not simply a promise to restore service. A team that sets an RTO without accounting for data restoration, available capacity and traffic routing may have a target its architecture cannot meet. Likewise, the RPO makes the potential cost of delayed or infrequent backups explicit: the more time between synchronized copies, the more recent data could be lost in a failure.
AI services add dependencies that can shift outside a software team’s direct control. If a model provider changes a model or becomes unavailable, service behavior or availability may be affected. The report’s emphasis on observability, multiple model or vendor options and routine vendor reviews reflects the need to detect and manage those dependencies. These are recommendations discussed in the source, not evidence that any specific AI system has failed or that one approach eliminates outages.
There is also a capacity trade-off. Keeping spare resources or maintaining a second live site can support faster recovery, but can add cost and operational complexity. The source argues that teams should match their recovery approach to their own requirements, while considering constraints such as scarce compute capacity.
As an affiliate, we earn on qualifying purchases.
From Site Outages to Model Risk
The article introduces the book as a practical work on distributed-system reliability and says its scope includes disaster recovery, infrastructure availability and AI-related design questions. Newman took three years to complete it, rather than the two he initially planned, according to the newsletter, which says he revisited the material to account for AI’s relevance to system design.
The excerpt uses “site” as a general term for the location of computing infrastructure, whether that is a public-cloud region, a private-cloud location or a traditional data center. That terminology lets the recovery patterns apply across different deployment setups. The newsletter says the book also discusses considerations involved in using multiple cloud vendors, but the supplied excerpt does not provide those details.
The Pragmatic Engineer identifies the text as an excerpt from Building Resilient Distributed Systems, published by O’Reilly Media in 2026 and reproduced with permission. The report’s author says the recommendation was not paid and that the links were not affiliate links.
“The recovery point objective (RPO) defines the maximum amount of time that can have passed since the data was last backed up or synchronized.”
— Sam Newman, in the book excerpt
distributed system failover solutions
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Recovery Details Still Depend on Teams
The supplied material does not give specific RPO or RTO targets, cost estimates or test results for the recovery patterns. It also does not compare the patterns against one another for particular workloads, so it cannot establish which option is appropriate for a given organization.
The newsletter summarizes the book’s discussion of AI reliability, but the provided excerpt does not include the full relevant chapter or named examples of model-provider outages, model changes or vendor reviews. It is also unclear how common the cited GPU and CPU shortages are across regions or providers, or what capacity any particular team can secure during a broad disruption. The material offers planning guidance, not a report of a specific outage or measured industry-wide trend.
As an affiliate, we earn on qualifying purchases.
Teams Must Set Their Own Targets
The next practical step for teams is to establish acceptable data loss and downtime, then test whether their chosen recovery setup can meet those targets. That assessment needs to include data synchronization, alternate-site capacity and traffic redirection, rather than treating failover as a single action. Teams building AI-powered products can also review model and infrastructure dependencies, monitoring coverage, vendor terms and the feasibility of fallback options.
The source does not announce a follow-up milestone, deployment or policy change. The book is available from its publisher, according to the report; no further release details or scheduled updates are provided.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is the news?
The Pragmatic Engineer published excerpts from Sam Newman’s 2026 book Building Resilient Distributed Systems, including material on disaster recovery and AI-system reliability.
What do RPO and RTO mean?
RPO sets the maximum period of data loss a team is prepared to accept. RTO sets the maximum acceptable period that a system remains unavailable after an incident.
Which recovery approaches does the excerpt name?
It lists backup and restore, pilot-light failover, warm failover and multisite active-active operation. The options differ in cost, complexity and recovery capability.
What AI reliability issues does the report identify?
The newsletter summary points to model-provider resilience, models changing, observability and infrastructure availability. It says the book discusses using multiple models or vendors, transparent costs and frequent vendor reviews as ways to manage uncertainty.
Does the source name a best recovery strategy?
No. It presents a range of patterns and says teams need to set requirements for their own systems. The supplied material does not give universal RPO or RTO values or identify one approach as best for every organization.
Source: rss
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
