Amazon S3

Technical details

Design

Amazon S3 manages data with an object storage architecture which aims to provide scalability, high availability, and low latency with high durability. The basic storage units of Amazon S3 are objects which are organized into buckets. Each object is identified by a unique, user-assigned key. Buckets can be managed using the console provided by Amazon S3, programmatically with the AWS SDK, or the REST application programming interface. Objects can be up to five terabytes in size. Requests are authorized using an access control list associated with each object bucket and support versioning which is disabled by default. Since buckets are typically the size of an entire file system mount in other systems, this access control scheme is very coarse-grained. In other words, unique access controls cannot be associated with individual files. Amazon S3 can be used to replace static web-hosting infrastructure with HTTP client-accessible objects, index document support, and error document support. The Amazon AWS authentication mechanism allows the creation of authenticated URLs, valid for a specified amount of time. Every item in a bucket can also be served as a BitTorrent feed. The Amazon S3 store can act as a seed host for a torrent and any BitTorrent client can retrieve the file. This can drastically reduce the bandwidth cost for the download of popular objects. A bucket can be configured to save HTTP log information to a sibling bucket; this can be used in data mining operations.

Various User Mode File System (FUSE)–based file systems for Linux or other Unix-like operating systems can be used to mount an S3 bucket as a file system. The semantics of the Amazon S3 file system are not that of a POSIX file system, so the file system may not behave entirely as expected.{{cite web |access-date = 2012-06-29 |archive-url = https://web.archive.org/web/20120805030057/http://code.google.com/p/s3ql/wiki/other_s3_filesystems |archive-date = 2012-08-05 |url-status = live

Amazon S3 storage classes

Amazon S3 offers nine different storage classes with different levels of durability, availability, and performance requirements.

• Amazon S3 Standard is the default. It is general purpose storage for frequently accessed data.
• Amazon S3 Express One Zone is a single-digit millisecond latency storage for frequently accessed data and latency-sensitive applications. It stores data only in one availability zone.

The Amazon S3 Glacier storage classes above are distinct from Amazon Glacier, which is a separate product with its own APIs.

File size limits

An object in S3 can be between 0 bytes and 5 TB. If an object is larger than 5 TB, it must be divided into chunks prior to uploading. When uploading, Amazon S3 allows a maximum of 5 GB in a single upload operation; hence, objects larger than 5 GB must be uploaded via the S3 multipart upload API.

Scale

As of 2025, S3 stores 500 trillion objects, 100s of exabytes of data, serves 200 million requests per second, and peaks at about 1 petabyte per second in bandwidth.

Uses

Notable users

• Photo hosting service SmugMug has used Amazon S3 since April 2006. They experienced a number of initial outages and slowdowns, but after one year they described it as being "considerably more reliable than our own internal storage" and claimed to have saved almost $1 million in storage costs.
• Netflix uses Amazon S3 as their system of record. Netflix implemented a tool, S3mper, to address the Amazon S3 limitations of eventual consistency. S3mper stores the filesystem metadata: filenames, directory structure, and permissions in Amazon DynamoDB.
• Reddit is hosted on Amazon S3.
• Bitcasa, and Tahoe-LAFS-on-S3, among others, use Amazon S3 for online backup and synchronization services. In 2016, Dropbox stopped using Amazon S3 services and developed its own cloud server.
• Swiftype's CEO has mentioned that the company uses Amazon S3.

S3 API and competing services

The broad adoption of Amazon S3 and related tooling has given rise to competing services based on the S3 API. These services use the standard programming interface but are differentiated by their underlying technologies and business models. A standard interface enables better competition from rival providers and allows economies of scale in implementation, among other benefits. Users are not required to go directly to Amazon as several storage providers such as Cloudian, Backblaze B2, Wasabi offer S3-compatible storage with options of on-premises and private cloud deployments. Several open-source implementations of S3-compatible object store servers can be self-hosted, such as MinIO and Garage. IBM Cloud Object Storage also offers a cloud-based S3-compatible option of Object Storage.

History

Amazon Web Services introduced Amazon S3 in 2006.

In November 2017, AWS added default encryption capabilities at bucket level.{{Cite web|url=https://www.youtube.com/watch?v=NXehLy7IiPM&list=PL2yQDdvlhXf_ZsP25dGLTNbrVSphM2JDl&index=666|title = AWS re:Invent 2024 - Dive deep on Amazon S3 (STG302)

References

Citations

Sources

• {{cite web | access-date = 2014-12-23 | archive-date = 2014-12-23 | archive-url = https://web.archive.org/web/20141223092055/http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.html | url-status = live
• {{cite web
• {{cite web
• {{cite web |access-date = 2009-01-07 |archive-url = https://web.archive.org/web/20081103070950/http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1014 |archive-date = 2008-11-03 |url-status = dead

References

1. (2017-09-08). "Mobile Cloud Computing: Foundations and Service Models". Morgan Kaufmann.
2. "Cloud Object Storage - Store & Retrieve Data Anywhere - Amazon Simple Storage Service".
3. (2006-03-14). "Amazon Web Services Launches "Amazon S3"".
4. "5 Key Events in the history of Cloud Computing - DZone Cloud". dzone.com.
5. (2007-11-06). "Amazon Web Services Offers European Storage for Amazon S3".
6. (18 April 2013). "Amazon S3 – Two Trillion Objects, 1.1 Million Requests / Second - Amazon Web Services".
7. (2019-10-16). "What is Cloud Object Storage? – AWS".
8. "Tech Blog » Starting Websphere in Cloud and saving the data in S3".
9. "open-guides/og-aws".
10. "Error Responses - Amazon Simple Storage Service".
11. "Using versioning in S3 buckets - Amazon Simple Storage Service".
12. "Introduction to Amazon S3 - Amazon Simple Storage Service".
13. "How to use Amazon S3 for Web Hosting".
14. [http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?WebsiteHosting.html Amazon Simple Storage Service] {{Webarchive. link. (2011-02-20 Docs.amazonwebservices.com. Retrieved on 2013-08-09.)
15. http://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.html {{Webarchive. link. (2014-12-23 Server Access Logging)
16. Craig-Wood, Nick. "rclone mount".
17. (2025-11-15). "awslabs/mountpoint-s3". Amazon Web Services - Labs.
18. "Cloud Storage Classes – Amazon Simple Storage Service (S3) – AWS".
19. (2023-11-28). "Announcing the new Amazon S3 Express One Zone high performance storage class {{!}} AWS News Blog".
20. (June 21, 2022). "How to Upload Large Files to S3".
21. Garman, Matt. "AWS re:Invent 2025 - Keynote with CEO Matt Garman". Amazon.
22. (5 Dec 2024). "AWS re:Invent 2024 - Dive deep on Amazon S3 (STG302)".
23. (November 10, 2006). "Amazon S3: Show Me the Money". SmugMug.
24. "S3mper: Consistency in the Cloud".
25. "Introduction to Amazon S3".
26. Hern, Alex. (2017-02-02). "Amazon Web Services: the secret to the online retailer's future success".
27. (2015). "AWS Case Study: reddit". aws.amazon.com.
28. (May 16, 2013). "Bitcasa Legal".
29. (August 21, 2012). "What is Tahoe-LAFS-on-S3?".
30. "The Epic Story of Dropbox's Exodus From the Amazon Cloud Empire".
31. (2018-02-23). "Dropbox saved almost $75 million over two years by building its own tech infrastructure". GeekWire.
32. (July 1, 2013). "Swiftype Explains Their Cloud Stack".
33. Watters, Audrey. (12 July 2010). "Cloud Community Debates, Is Amazon S3's API the Standard? (And Should It Be?)". SAY Media, Inc..
34. (1990). "Crossroads of Information Technology Standards". The National Academies Press, 1990..
35. "How to use S3-compatible storage {{!}} TechTarget".
36. Yegulalp, Serdar. (2017-01-31). "Google Go-powered object storage server offers open source AWS alternative".
37. Bhutani, Dhruv. (2025-10-27). "Stop using MinIO, and use this free, user-friendly tool instead".
38. "IBM Cloud Object Storage S3 API {{!}} Introduction on IBM Cloud API Docs".
39. Overview of Amazon Web Services, 2018, https://docs.aws.amazon.com/whitepapers/latest/aws-overview/introduction.html {{Webarchive. link. (2017-11-18)
40. link. (2018-07-29)
41. Vogels, Werner. (2008-03-19). "Happy Birthday, Amazon S3!". All Things Distributed.
42. (31 March 2009). "Celebrating S3's Third Birthday With Special Anniversary Pricing - Amazon Web Services".
43. "Amazon's Head Start in the Cloud Pays Off".
44. (9 March 2010). "Amazon S3 Now Hosts 100 Billion Objects".
45. (14 March 2021). "Celebrate 15 Years of Amazon S3 with 'Pi Week' Livestream Events".
46. (14 March 2023). "Celebrate Amazon S3's 17th birthday at AWS Pi Day 2023".
47. (20 December 2024). "Adapting to change with data patterns on AWS: The "aggregate" cloud data pattern | AWS Storage Blog".