Track: Architectures you've always wondered about

Abstract: Amazon's Simple Storage Service (S3) provides a web services interface to durably store and retrieve any amount of data. S3 servers are located on multiple continents and across many different data center facilities. From time to time these data centers or components within the data centers fail in unusual and spectacular ways. A rigorously designed and implemented system makes these failures largely unnoticeable to our users. This talk will focus on the approaches taken by S3 in architecting our systems to survive component, machine, and data center failures. We will begin by categorizing and classifying the different types of failures and evaluating which of these we are most concerned about. We will discuss some general design principles used by S3 to mitigate failures. We will conclude with a description of specific real-world failures and how the design of the system either mitigated the failure or compounded it.