Koshy John

Leveraging Deterministic Updates to Improve the Customer Experience on Debian

Every change to a production system carries risk, and this risk is magnified when applications are distributed across hundreds or thousands of hosts. Security updates to Debian are published on an as-needed basis with no ability to predict in advance what package will change at any given moment, which means that an update from the repositories is nondeterministic over time. The apt update/apt upgrade operation performed today may behave differently from the one performed yesterday.

In this talk we present a deterministic update strategy based on snapshot support introduced with apt 2.7.0. Using apt snapshots, administrators can lock their systems to a specific point-in-time view of the Debian package repositories backed by snapshot.debian.org. This approach provides repeatable and deterministic update behavior with a number of safety benefits that the administrator can incorporate into their infrastructure testing and deployment strategy in a variety of ways:

• Pre-production testing that reflects what’s subsequently going to be deployed to production

• The ability to execute phased updates following a ring-based deployment pattern

Building on this foundation, Microsoft has added support for Debian within Microsoft Azure Guest Patching Service. We describe this service and how it can be used to safely and reliably manage fleets of any size within the Microsoft Azure cloud computing environment; while providing the capabilities listed above.

Azure’s Safe Deployment Principles monitor the rollout of an update on VMs. Azure pauses a rollout and pushes a new update if a regression is detected on a VM. The same safety mechanism will be in place with Debian Snapshots.

By pinning an update for a customer’s fleet across regions, Azure is simplifying the way customers keep their assets secure through Debian Snapshots.