Earlier in October, Amazon Web Services, Inc. (AWS), announced two
new capabilities to help customers more quickly and cost-effectively transfer data
of all types and sizes to the AWS Cloud. AWS Snowball is a petabyte-scale data
transport appliance that can securely transfer 50 TB per appliance of data into and
out of AWS. Amazon Kinesis Firehose is a fully managed service for loading
streaming data into AWS.
Today, customers are moving an increasing number of applications and large
volumes of data to the AWS Cloud – everything from app log files to digital media,
genomes, and petabytes of sensor data from connected devices. While AWS Direct
Connect provides customers with a dedicated, fast connection to the AWS network, AWS Snowball and Amazon Kinesis Firehose are ideal for customers that need to transfer data in large batches, have data located in distributed locations, or require continuous loading of streaming data. With AWS Snowball and Amazon Kinesis Firehose, AWS customers now have two new ways to more easily and cost-
effectively get large data sets and streaming data into the AWS Cloud.
“It has never been easier or more cost-effective for companies to collect, store,
analyse, and share data than it is today with the AWS Cloud,’ said Bill Vass, vice
president, AWS Storage Services. “As customers have realised that their data
contains key insights that can lead to competitive advantage, they’re looking to get
as much data into AWS as quickly as possible. AWS Snowball and Amazon Kinesis
Firehose give customers two more important tools to get their data into AWS.’
AWS Snowball
Customers who need to transfer large amounts of data to AWS face a challenge –
the time it takes to upload data. For instance, if a company committed 100
megabits per second of their total bandwidth capacity to transferring data to AWS,
transferring 100 TB of data via that connection would take about 100 days.
Companies could choose to spend more money expanding their bandwidth capability
or upgrading their network, but most don’t want to do so simply to support sending
more data to the cloud. Now, AWS provides a solution to this problem with AWS
Snowball – a durable and tamper-resistant, encrypted, and portable storage
appliance that customers can use to move that same 100 TB of data to AWS in less
than a week, and at as little as one-fifth of the cost of using high-speed internet.
Customers create a job using the AWS Management Console, AWS ships the
appliance directly to the customer, and the customer, upon receiving the appliance,
simply plugs it into their local network. AWS Snowball provides a simple data
transfer client which customers use to encrypt and transfer 50TB of data to each
appliance. Customers can use multiple AWS Snowball appliances in parallel to
transfer larger data sets within the same time frame. Once a customer’s data is
completely loaded onto an AWS Snowball, its E Ink shipping label is automatically
updated with the AWS shipping address, and customers can track the status of the
transfer job using Amazon Simple Notification Service (SNS), text messages, or the
AWS Management Console.
AWS Snowball appliances use multiple layers of security to protect customer data.
In addition to tamper-resistant enclosures, AWS Snowball also employs end-to-end,
256-bit encryption, along with an industry-standard Trusted Platform Module (TPM)
designed to ensure both security and full chain-of-custody for customer data. Once
a customer’s data has been transferred from the AWS Snowball to AWS’s data
stores (initially Amazon S3), AWS erases all data from the AWS Snowball appliance,
following the standards defined by the National Institute of Standards and
Technology (NIST) guidelines for media sanitisation. Customers can get a report
and confirm that all of their data has successfully been loaded into AWS before
deleting the local copy of their data.
Amazon Kinesis Firehose
Mobile devices, software applications and services, wearables, industrial sensors,
and IT infrastructure generate staggering amounts of data – sometimes TBs per
hour. In 2013, AWS introduced Amazon Kinesis Streams to allow customers to build
applications that collect, process, and analyse this streaming data with very high
throughput. Many customers use Amazon Kinesis Streams to capture streaming data
and load it into Amazon S3 or Amazon Redshift. Until now, this required customers
to manage the Amazon Kinesis data streams and write custom code to load the
data. Now, Amazon Kinesis Firehose makes this as easy as an API call. Amazon
Kinesis Firehose captures data from hundreds of thousands of different sources and
loads it directly into AWS, in real-time. Customers create an Amazon Kinesis
Firehose Delivery Stream in the AWS Management Console and specify the target
Amazon S3 bucket or Amazon Redshift table, and the time frequency at which they
want fresh data delivered to the destination. Customers can also configure Amazon
Kinesis Firehose to batch, compress, and encrypt streaming data before delivery at
specified time intervals.
