AWS S3 Buckets
Published on 01/13/2019
3 min read
- a safe place to store your files
- object based storage.
"objects" are "flat files", i.e. pdf's, photos, videos
- Note, there also exists the concept of "block based storage", where you would install an operating system (i.e. EC2 container)
- data spread across multiple devices and facilities.
- designed to withstand failure.
- File size limit a 5 TB.
- Unlimited Storage (grows with your needs)
- Files are stored in "Buckets" (aka "folders")
- S3 is a universal namespace, thus, names MUST BE UNIQUE GLOBALLY.
- Whenever you create a bucket, a http link of the following format is created:
- more generally: https://
- more generally: https://
- "read after write consistency for PUTS of new Objects" and "Eventual consistency for overwrite PUTS and DELETES"
- When you upload a file, you will ALWAYS receive a HTTP 200 code upon success.
- After you upload a file, you can read it immediately, but, there is a delay for deleting and updating because changes need to propagate over aws infrastructure.
- bucket name MUST BE UNIQUE.
- each file within a bucket will have it's own unique HTTP address.
- by default, public access to files within a bucket is TURNED OFF. You will receive an 'access denied' message.
you can make individual objects within buckets public through the GUI.
- Owner Access
- what can the owner of the account which they are logged into do.
- Access for Other AWS accounts
- Any accounts that have been granted permission other than the creator
- Public Access
- permissions for the world at large/anyone with a link to the bucket/object.
Individual objects DO NOT inherit
- Storage class, change redundancy and access allotments for cost savings
- Encryption, optional, defaults to off.
- Metadata, key-value pairs
- Tags, key-value pairs
- if enabled, versioning CANNOT be turned off. (though, it CAN be suspended)
- Each time a file is replaced or edited, a unique, standalone file will be saved.
- this is a bad idea for very large files that change often.
- you can inspect each version by clicking on an object. There will be a "latest version" dropdown menu near the top of the dialog box that pops up.
- if you DELETE an object, it will NOT delete the version history
- when inspecting your bucket, there is a "versions" toggle to hide/show multiple versions of the same file.
- creating buckets hosted on servers that live in different geographical areas.
- versioning needs to be turned on in both buckets.
Aexists with three files, then
Bis created to be a replica, ONLY new files will be replicated in
B. The existing files will not be replicated upon
very low-cost storage service that provides secure, durable, and flexible storage for data backup and archives.
can be used for Lifecycle Management
The basic gist is that you can set a schedule for your files uploaded to a s3 bucket to live and die by.
i.e. hello.txt is uploaded, 30 days later it's moved to infrequent access storage, 60 days later it's moved to Glacier storage, 360 days later it's deleted, PERMANENTLY, from Glacier.