- Universal across regions
- New users have 0 permissions
- Users: End Users
- Groups: A collection of users under on set of permissions
- Roles: Created and assigned to AWS resources
- Policies - Document that defines one or more permissions
- A way to grant permissions to entities you trust.
AWS Storage and CDN - S3, Glacier and CloudFront
Simple Storage Service - secure, durable, highly-scalable object storage.
- 0 - 5 TB file size
- Unlimited storage
- Universal namespace, must be unique
- 200 HTTP Status code on successful upload
- Host static sites/content
- Data consistency model
- Read after write consistency for PUTS of new Object
- Eventual Consistency for overwrite for PUTS and DELETES (can take some time to propagate)
- 99.99% availability 99.9% SLA
- 11X9’s durability (99.999999999%)
- Designed to survive the loss of 2 facilities concurrently
Simple key value store
- Key: file name
- Value: object data
- Version ID
- Subresources: Access Control Lists, Torrent
Cross Region Replication
- Only applies to new or modified files
- Use command lines to transfer
- Versioning must be enabled
- Regions must be unique
- No Daisy chaining
- Delete markers are replicated
- Deleting individual versions or delete markers are not replicated
- S3 -IA Infrequently Accessed (lower fee)
- S3 Reduced Redundancy Storage (slightly less durability)
- Storage Management Pricing (tags)
- Data Transfer Pricing
- Transfer Acceleration: Uses CloudFront to move data long distances
- Allows multi factor authentication deletes
Once enabled, can only be suspended not removed
- Can be used w/ versioning
- Can be applied to current and previous versions
- Transition to IA Storage (min 128Kb and min 30 days)
- Archive to Glacier (min 60 days)
- Permanently delete
Security and Encryption
Buckets are PRIVATE by default
Use bucket policies and access control lists to control access
S3 buckets can be configured to log access
*Types of encryption
- In Transit: SSL/TLS
- At Rest
- Server Side:
- S3 Managed Keys - SSE-S3
- AWS Key Management Service, Manage Keys - SSE-KMS
- Server Side Encryption with Customer Provided Keys - SSE-C
- Client Side Encryption
Utilizes edge location to accelerate uploads to S3.
Read the S3 FAQ
(Archival), $0.01 per gb, retrieval = 3-5 hours, 90 minimum storage time
Content Delivery Network
Content spread out to edge locations
- Not just read only
- Objects are cached for TTL (Time To Live) 24hr default
- You can clear the cache, but it costs
- Separate from AWS Regions/AZs
This is where the files are stored that are to be distributed by the CDN
The collection of edge locations
- Web Distribution: Websites
- RTMP: Serving media content
On premise data storage to AWS cloud (S3)
VM on premise server
- File Gateway (NFS)
- Files in S3
- Nothing on premise
- Volumes Gateway (iSCSI) SQL server
- Virtual hardiest (EBS snapshots)
- Stored volumes
- Asynchronous backups to S3 in an EBS Snapshot (1GB - 16TB)
- Entire dataset on premise
- Cached volumes
- Complete data on S3 (1GB -32TB)
- Recently read data on premise
- Do not need as much storage on premise
- Tape Gateway (VTL)
- Virtual tape
- NetBackup, Backup Exec, Veeam…
- Petabyte scale data transport solution 80TB
- Trusted Platform Module
- 256-bit encryption
- Tamper resistant enclosures
- 1/5th the cost of data transfer
- Snowball Edge
- 100TB data transfer
- Has compute capacity
- Can perform Lambda functions
- Datacenter in a box
- Petabyte or Exabytes of data
- Shipping container 100PB
EC2 - Elastic Compute Cloud
Allows developers the ability to pay for only the capacity you use/need.
- On Demand - allow you to pay a fixed rate by the hour (or by the second)
- Reserved 1 to 3 year contract
- Spot - set bid a price for instance capacity
- Dedicated Hosts - Physical EC2 server dedicated to your use.
EC2 Instance Types
||Memory Intesive Apps/DBs
||CPU intesive Apps/DBs
||Video encoding/ 3D application streaming
||High speed storage
||NoSQL DBs, data warehousing
||Field programmable gate array
||Hardware accelleration for your code
||Lowest Cose, General purpose
||Web servers, small DBs
||Graphics/General purpose GPU
||Machine leaerning, mining
||SAP HANA, Apache Spark
DR MC GIFT PX
- D: Density
- R: RAM
- M: Main general purpose
- C: Compute
- G: Graphics
- I: IOPS (Databases)
- F: FPGA
- T: Tiny, general purpose
- P: Pictures, moving pictures, general GPU
- X: X-treme memory optimized
EBS - Elastic Block Store
Block based storage disks that you can attach to your EC2 instances.
Automatically replicated for you mitigating hardware failure. (In same AZ)
- General Purpose SSD (GP2)
- Balance of price and performance
- 3 IOPS per GB up to 10,000 IOPS w/ bursts up to 3000 IOPS
- Provisioned IOPS SSD (IO1)
- Designed for I/O intensive applications/databases
- Only needed if you require over 10,000 IOPS
- Up to 20,000 IOPS
- Throughput Optimized HDD (ST1)
- Big data, data warehousing, log processing
- Cannot be a boot volume
- Cold HDD (SC1)
- Lowest cost for infrequently accessed data
- File server
- Cannot be a boot volume
- Magnetic (Standard)
- Lowest storage cost per GB of all EBS volume types that are bootable
Need to Know
- The difference between:
- On demand
- Dedicated Hosts
- Sport instances:
- If you terminate, you pay for the hour.
- If AWS terminates the spot instance, you get the hour it was terminated for
- EBS types and applications
- You can not mount 1 EBS volume to multiple EC2 instances, use EFS instead
for shared storage.
EC2 Lab Tips
- Use tags to reduce costs!
- Cannot encrypt root volume (Default AMI), only encrypt EBS attached storage via AWS.
- Termination protection, off by default.
- Root EBS valume deleted when instance is terminated.
It is a virtual firewall.
- Multiple Security Groups can be assigned to resources.
- All inbound traffic blocked by default
- All outbound traffic allowed
- Security Groups are STATEFUL
- Inboude rules that allow traffic in automatically allow traffic out. (state)
- Differs from ACL
- Cannot not block specific IP addresses. Use ACL.
- Cannot specify deny rules, only allow.
EBS Volumes - Lab
Upgrading and downgrading EC2 EBS volumes.
- EC2 instance must always be in the same availability zone as the EBS Volume.
- You can modify gp2 on the spot without stopping the instance to IO
- Cannot modify standard.
- Move data to another availability zone/region -> create a snapshot -> copy snapshot to new region -> create new image on a new EBS Volume.
- Snapshots exist on S3 (not user accessible).
- Incremental, only modified blocks are moved to s3.
- Taking a snapshot of a root device can cause performance issues while
- You can create AMI’s from both Volumes and Snapshots.
- Snapshots of encrypted volumes are encrypted automatically
- Snapshots can be shared if they are unencrypted
Raid, Volumes & Snapshots
- Redundant Array of Independent Disks
- RAID 0 - Striped, No Redundancy, Good Performance.
- RAID 1 - Mirrored, Redundancy
- RAID 5 - Good for reads, bad for writes, AWS does not recommend ever puttin RAID 5’s on EBS!
- RAID 10 - Striped & Mirrored, Good Redundancy, Good Performance.