In this blog post, we will cover the AWS Storage overview and types of storage options offeredwhat are they intended for?
AWS storage services are low-cost data storage with high durability and high availability. You also get the option to backing up information, archiving, and disaster recovery.
Cloud Storage Overview
Cloud storage is a cloud computing model that stores data on the Internet through a cloud computing provider that manages and operates data storage as a service. It’s delivered on-demand with just-in-time capacity and costs and eliminates buying and managing your own data storage infrastructure.
Also Read: Our Blog post on AWS Secrets Manager
There Are 3 Types of Cloud Storage
1. Object Storage – Applications developed within the cloud often cash in on object storage’s vast scalability and metadata characteristics. Object storage solutions like Simple Storage Service (Amazon S3) and Amazon Glacier are ideal for building modern applications from scratch that need scale and adaptability, and may even be wont to import existing data stores for analytics, backup, or archive.
2. File Storage – Many applications got to access shared files and need a filing system. this sort of storage is usually supported with a Network Attached Storage (NAS) server. File storage solutions like Elastic File System (Amazon EFS)are ideal to be used in cases like large content repositories, development environments, media stores, or user home directories.
3. Block Storage – Other enterprise applications like databases or ERP systems often require dedicated, low latency storage for every host. this is often analogous to direct-attached storage (DAS) or a cargo area Network (SAN). Block-based cloud storage solutions like Elastic Block Store (Amazon EBS) and EC2 Instance Storage
Storage Offered By Amazon Web Services (AWS)
Simple Storage Service (Amazon S3)
Amazon S3 the oldest and most supported storage platform of AWS uses an object storage model that is built to store and retrieve any amount of data. Data can be accessed everywhere such as websites, mobile apps, corporate applications, and data from IoT sensors or devices that can be dumped onto S3.
S3 stands for Simple Storage Service. It is secure, durable, and highly scalable object storage which can store different types of files. It’s a static storage service useful for static website hosting, media distribution, version management, big data analytics, and archiving. It is easy to use with a simple web interface to store and retrieve any amount of data from anywhere on the web. Files are stored in Buckets (similar to folders) and bucket names must be unique. Files can be from 0 bytes to 5TB.
Usage
S3 has been highly used for hosting web content with support for high bandwidth and demand. Scripts can also be stored in S3 making it possible to store static websites that use JavaScript.It supports the migration of data to Amazon Glacial for cold storage, by using lifecycle management rule for data stored in S3.
Features
Amazon S3 runs upon the world’s largest global cloud infrastructure and was built from the ground up to deliver a customer promise of 99.999999999% durability. Data is automatically distributed across a minimum of three physical facilities that are geographically separated within an AWS Region, and also automatically replicates data to any other AWS Region.
Security
S3 supports 3 forms of encryption, including server-side-encryption and client-side-encryption. Data in S3 can only be accessed by other users or AWS account when they have been granted access by the admin by writing the access policy. With the support of Multi-Factor Authentication (MFA) another layer of security can be added for object operation.S3 supports multiple security standards and compliance certifications.
Benefits Of S3
- It is the most supported cloud storage service. Amazon S3 stores and protects your data by working with a partner from the AWS Partner Network (APN) the largest community of technology and consulting cloud services providers.
- Save costs without sacrificing performance by storing data across the S3 Storage Classes, thus a wide range of cost-effective storage classes.
- S3 is a highly scalable secure and durable (99.999999999% data durability) storage type offered by AWS.
- S3 is the only object storage service that allows you to block public access to all of your objects at the bucket or the account level with S3 Block Public Access.
- S3 Access Points make it easy to manage data access with specific permissions for your applications using a shared data set
- Improves query performance by using S3 Select to retrieve subsets of object data, instead of the entire object.
Read More: About AWS SNS
Amazon Glacier
Amazon Glacier is a secure, durable, and extremely low-cost storage service for data archiving and long-term storage. Glacier allows you to run powerful analytics directly on archived data. The glacier can also utilize other AWS storage services such as S3, CloudFront, etc. to move data in and out seamlessly for better and effective results.
Usage
Amazon Glacier is used to stores data in the form of archives. An archive can consist of a single file, or a combination of several files as a single archive, and the archives are organized in vaults. With support to querying on data to retrieve a particular subset of data that you need from within an archive.
Feature
Since AWS Glacier is an archiving service, durability must be of utmost priority. The glacier is designed to provide average annual durability of 99.999999999% for archives. Data is automatically distributed across a minimum of three physical facilities that are geographically separated within an AWS Region.
Security
Initially, Glacial data can only be access by the account owner/admin, however, access control can be set up for other people by defining access rules in AWS Identity and Access Management (IAM) service. Glacier uses server-side encryption to encrypt all data. Lockable policies can be defined to lock vaults for long-term records retention.
Elastic File System (Amazon EFS)
As the name suggests EFS delivers a scalable, elastic, highly available, and highly durable network file system as-a-service. The storage capacity of EFS is elastic and is capable of growing and shrinking automatically depending on the requirement. EFS supports Network File System versions 4 (NFSv4) and 4.1 (NFSv4.1).
Amazon EFS is an NFS file system service offered by AWS. An Amazon EFS file system is excellent as a managed network file system that can be shared across different Amazon EC2 instances. Amazon EFS works like NAS devices and performs well for big data analytics, media processing workflows, and content management. EFS can be accessed by multiple instances at a time through NFS protocol. It is used as a clustered database and document sharing.
Usage
EFS is a network file system that can expand to petabytes with parallel access from EC2 instances. Elastic File System EFS is mounted on Amazon EC2 instances. Multiple EC2 instances can even share a Single EFS file system, allowing access to the large application that grew beyond a single instance. EFS can be mounted on-premises data center connected with Amazon Virtual Private Cloud (VPC) with AWS Direct Connect service.
Feature
EFS stores data as objects and each object is stored across multiple availability zones within a region. More durable than S3. Ability to make API calls.
Security
There are three main levels of access controls to consider when it comes to the EFS file system.
- IAM permissions for API calls
- Security groups for EC2 instances and mount targets
- Network File System-level users, groups, and permissions.
AWS allows connectivity between EC2 instances and EFS file systems. You can associate one security group with an EC2 instance and another security group with an EFS mount target associated with the file system. These security groups act as firewalls and enforce rules that define the traffic flow between EC2 instances and EFS file systems.
Benefits of EFS
- With EFS you need not worry about managing file servers or storage, updating hardware, configuring software, or performing backups as EFS is a fully managed service
- The distributed architecture of Amazon EFS provides data protection from an AZ outage, system and component failures, and network connection errors.
- Network Access to the files can be controlled using Virtual Private Cloud security group rules and with Identity Access Management policies and EFS access points you can control the access to your files
- Amazon EFS is designed to provide the throughput, IOPS, and low latency needed for a broad range of workloads
- With Amazon EFS, storage capacity is elastic, growing and shrinking automatically as you add and remove files, dynamically providing the storage capacity to applications as they need it.
- AWS EFS provides encryption of data both at rest and in transit so that your data is secure.
Elastic Block Store (Amazon EBS)
Similar to EFS, EBS volumes are network file systems. Volumes get automatically replicated within Availability Zones for high availability and durability.
Amazon Elastic Block Store (EBS) is an easy to use, high-performance block storage service designed for use with Amazon Elastic Compute Cloud (EC2) for both throughput and transaction-intensive workloads at any scale. A broad range of workloads, such as relational and non-relational databases, enterprise applications, containerized applications, big data analytics engines, file systems, and media workflows are widely deployed on Amazon EBS.
Usage
It is durable block-level storage to be used with EC2 instances in the AWS cloud. EBS Volumes are used by mounting them onto EC2 instance as you will do with a physical hard drive on-premise and then format the EBS volume to the desired file system. EBS allows for dynamically increasing capacity, performance tuning and you can even change the type of volume with any downtime or performance impact.
Features
EBS allows for save point-in-time snapshots of volumes to increases the durability of the data stored. Each separate volume can be configured as EBS General Purpose (SSD), Provisioned IOPS (SSD), Throughput Optimized (HDD), or Cold (HDD) as needed.EBS Volumes has a very low failure rate of about 0.1 to 0.2 percent.
Security
IAM policy is needed to be defined to allow access to EBS volumes. Coupled with encryption for data-at-rest and data-in-motion security it offers a strong defense-in-depth security strategy for your data.
Benefits Of EBS
- EBS volumes function well even if the load is very high. SSD backed options include a volume designed for high-performance applications and a general-purpose volume that offers strong price performance for most workloads.
- EBS volumes are designed to protect against failures by replicating within the Availability Zone (AZ), offering 99.999% availability.
- EBS provides six different volumes at different price points enabling you to optimize cost and choose volumes according to your needs.
- Elastic Volumes capability allows you to increase storage, tune performance up and down, and change volume types without any disruption to your workloads
- EBS volumes can be encrypted by default with a single setting in your account. EBS volumes support encryption of data at-rest, data in-transit, and all volume backups.
- EBS enables you to increase storage without any disruption to your critical workloads.
EC2 Instance Storage
EC2 Instance storage provides temporary block-level storage for EC2 instances.
Usage
Instance storage volumes are ideal for the temporary storage of data that changes frequently like buffers, queue caches, and scratch data. It can only be employed by one EC2 instance meaning volumes can’t be detached and attached to a different instance.
Features
Uses SSDs to deliver high random I/O performance, not intended to be used as durable disk storage. Data durability is provided through replication, or by periodically copying data to durable storageData on EC2 volume only persist during the lifetime of EC2 instance that it’s been related to
Security
IAM policy is required to be defined to permit secure control to users for performing operations like launch and termination of EC2 instances. When you stop or terminate an instance, the applications and data are erased and thus making the info inaccessible to a different instance within the future.
Also Read: Our blog post on AWS WAF.
Amazon FSx
Amazon FSx is a completely managed third-party file system solution. Amazon FSx utilizes SSD storage to provide fast performance with low latency.
It provides two file systems to choose from:
- Amazon FSx for Windows File Server
- Amazon FSx for Lustre
Usage
With the use of Amazon FSx, you can utilize the rich feature sets and fast performance of widely-used open source and commercially licensed file systems, while avoiding time-consuming administrative tasks like hardware provisioning, software configuration, patching, and backups. FSx provides cost-efficient capacity with high levels of reliability and integrates with a broad portfolio of AWS services to enable faster innovation.
Features
Amazon FSx provides a wide range of Solid-State Disk (SSD) and Hard Disk Drive (HDD) storage options enabling you to optimize storage price and performance for your workload requirements. It delivers sustained high read and writes speeds and consistent low latency data access.
Security
It automatically encrypts your data at rest using AWS KMS and in-transit using SMB Kerberos session keys. It is designed to meet the highest security standards and has been assessed to comply with ISO, PCI-DSS, and SOC compliance, and is HIPAA eligible.
Difference Between EFS, EBS, and S3
Now let’s take a look into the differences between Elastic File System, Elastic Block Storage, and Simple Storage Service
- Accessibility: S3 is publically accessible and you do not need a server to access it while EBS can only be accessed via EC2 machine and EFS can be accessed via several EC2 machines and AWS Services.
- Interface: S3 provides you a web interface while EBS provides a file system interface and EFS provides a web and file system interface.
- Storage Type: AWS S3 is object storage while Amazon EBS is block storage and Amazon EFS is file storage.
- Scalability: Amazon S3 and EFS are highly scalable while EBS is hardly scalable.
- Speed: Amazon S3 is slowest among the three and comes under tier 3 while Amazon EBS is fastest among the all and comes under tier 0 and Amazon EFS is faster than S3 and slower than EBS and comes under tier 1. So based on the performance hardware will be used in the backend of the storage services
- Backup: Amazon S3 is good for storing backups while EBS is meant to be an EC2 drive and since EFS is accessible over multiple servers so it’s good for shareable applications and workloads.
If you need help deciding which technology is the best fit for your unique business challenges and goals, I would suggest you to take our training program for AWS Solutions Architect.
Benefits Of AWS Storage
- No upfront cost it is a pay-as-you-go model.
- Worldwide access: You can access all your data worldwide just using an internet connection
- Storage can be increased or decreased with changes in data size.
- Low-cost data storage with high durability and high availability
- Plenty of choices for backing/archiving data in case of disaster recovery.
Over the years, Amazon Web Services (AWS) storage has been diversified vastly to cater to varying needs. With the vastly increasing data, new data storage technologies have transformed and are still evolving day by day.