Amazon S3 Tables deliver up to 3x faster query
performance and up to 10x higher transactions per second for
analytics workloads; Amazon S3 Metadata delivers queryable object
metadata in near real time to search, organize, and augment data to
accelerate data discovery
At AWS re:Invent, Amazon Web Services, Inc. (AWS), an
Amazon.com, Inc. company (NASDAQ: AMZN), today announced new Amazon
Simple Storage Service (Amazon S3) features that make S3 the first
cloud object store with fully-managed support for Apache Iceberg
for faster analytics and the easiest way to store and manage
tabular data at any scale. These new features also include the
ability to automatically generate queryable metadata, simplifying
data discovery and understanding to help customers unlock the value
of their data in S3.
- Amazon S3 Tables is the first cloud object store with built-in
Apache Iceberg table support and introduces a new bucket type to
optimize storage and querying of tabular data as Iceberg tables,
delivering up to 3x faster query performance, up to 10x higher
transactions per second (TPS), and automated table maintenance and
automation for analytics workloads.
- Amazon S3 Metadata streamlines data discovery in near real-time
by automatically capturing queryable object metadata, as well as
custom metadata using object tags, storing it in S3 Tables for
accelerating analytics across data lakes.
“As the leading object store in the world with more than 400
trillion objects, S3 is used by millions of customers, and we
continue to innovate to remove the complexity of working with data
at an unprecedented scale,” said Andy Warfield, vice president,
Storage, and distinguished engineer, AWS. “We have seen the rapid
rise of tabular data and, increasingly, customers want to query
across tables, improve query performance, and understand and
organize troves of data so they can easily find exactly what they
need. S3 Tables and S3 Metadata remove the overhead of organizing
and operating table and metadata stores on top of objects, so
customers can shift their focus back to building with their
data.”
S3 Tables and S3 Metadata are Apache Iceberg table-compatible so
customers can easily query their data using AWS analytics services
and open source tools, including Amazon Athena, Amazon QuickSight,
and Apache Spark.
Amazon S3 Tables—the easiest and fastest way to perform
analytics on Apache Iceberg tables in S3
Many customers today organize the data they use for analytics as
tabular data, most often stored in Apache Parquet, a file format
optimized for data queries. Parquet has become one of the fastest
growing data types in S3, and customers increasingly want to be
able to query these growing tabular data sets—often turning to open
table formats (OTF), an open source standard for storing data in
tables—because it helps organize, update, and track changes to
large amounts of data. Iceberg has become the most popular OTFs to
manage Parquet files, with customers using Iceberg to query across
billions of files containing petabytes or even exabytes of data.
However, Iceberg can be challenging for customers to manage as they
scale, often requiring dedicated teams to build and maintain
systems to handle table maintenance and data compaction, as well as
manage access control. These external systems are costly and
complex, and they require skilled teams to maintain, using up
valuable resources.
Amazon S3 Tables are purpose-built for managing Apache Iceberg
tables for data lakes. S3 Tables are specifically optimized for
analytics workloads, delivering up to 3x faster query performance
and 10x higher TPS compared to general purpose S3 buckets. S3
Tables automatically manage table maintenance tasks such as
compaction for better query performance and snapshot management to
continuously optimize query performance and storage costs, even as
customers’ data lakes scale and evolve. Customers can use S3 Tables
by creating a table bucket that optimizes the storage and querying
of tabular data in fully-managed Iceberg tables. With S3 Tables,
customers benefit from Iceberg capabilities like row-level
transactions, queryable snapshots via time travel functionality,
schema evolution, and more. In addition, S3 Tables provide
table-level access controls, allowing customers to define
permissions.
Genesys, a global leader in AI-powered experience orchestration,
plans to leverage Amazon S3 for its data lake. By utilizing S3
Tables' managed Iceberg support, Genesys expects to offer a
materialized view layer for its diverse data analysis needs. S3
Tables’ built-in support for Iceberg tables will simplify complex
data workflows by automating key maintenance tasks such as table
compaction, snapshot management, and unreferenced file cleanup.
Genesys is looking forward to improved performance and broad
support from Iceberg-compliant analytics tools that can read and
write Iceberg tables directly from S3. S3 Tables will be
foundational to Genesys' future data strategy, enabling the company
to deliver faster, more flexible, and reliable data insights to
support its AI-driven customer and employee experience
solutions.
Amazon S3 Metadata—the easiest and fastest way to discover
and understand data in S3
As more customers use S3 as their central data repository, the
volume and variety of data have grown exponentially, with metadata
becoming increasingly important as a way to understand and organize
large amounts of data so customers can find the exact objects they
need. To address this problem, many customers resort to building
and maintaining complex metadata capture and storage systems to
enrich their understanding of data. But these metadata systems are
expensive, time-consuming, and resource-intensive, often requiring
data engineers to manually track and update metadata as it flows
through their processing pipelines, as well as data analysts to
manually inspect massive object stores to find the specific data
they need for analytics and AI/ML data processing workflows.
Amazon S3 Metadata automatically generates queryable object
metadata in near real-time to help accelerate data discovery and
improve data understanding, eliminating the need for customers to
build and maintain their own complex metadata systems. S3 Metadata
lets customers query, find, and use data for business analytics,
real-time inference applications, and more. S3 Metadata
automatically generates object metadata, which includes
system-defined details like size and source of the object, and
makes it queryable via new S3 Tables. S3 Metadata updates object
metadata in S3 Tables as objects are added or removed, giving
customers an up-to-date view of their data. Customers can add their
own custom metadata using object tags to annotate objects with
information specific to their business, such as product SKUs,
transaction IDs, or content ratings, or with customer details.
Customers can easily query metadata using a simple SQL query,
enabling them to quickly find and prepare data for use in business
analytics and real-time inference applications, as well as
fine-tune foundation models, perform retrieval augmented generation
(RAG), integrate data warehouse and analytics workflows, perform
targeted storage optimization tasks, and more.
Organizations of all sizes are set to benefit from the data
discovery and understanding that S3 Metadata will bring. Roche, a
leading biotech company, plans to leverage S3 Metadata to
accelerate their future generative AI initiatives. As they develop
advanced large language model (LLM) applications like sophisticated
internal chatbots, they anticipate managing exponentially larger
volumes of unstructured data for enhanced RAG. S3 Metadata will
simplify the creation of a scalable metadata system, automatically
surfacing and updating metadata as new data is ingested. Roche
envisions using custom Lambda functions to extract complex,
business-specific metadata, integrating it seamlessly with S3
Metadata in a comprehensive Glue catalog. This will enable more
efficient organization and rapid identification of relevant
datasets for cutting-edge AI applications, allowing Roche to focus
on groundbreaking innovations in personalized healthcare.
Cambridge Mobile Telematics (CMT) is the world’s largest
telematics service provider. The company gathers sensor data from
devices and enriches it with contextual data to create a unified
view of vehicle and driver behavior that auto insurers, automakers,
commercial mobility companies, and the public sector use to power
risk assessment, safety, claims, and driver improvement programs.
CMT stores and analyzes multiple petabytes of data from millions of
IoT devices worldwide. As CMT scales, locating specific data for
developing new insights and models becomes increasingly
challenging. S3 Metadata, including system and custom metadata,
allows CMT to query petabytes of metadata, making finding relevant
data simple and cost-effective.
S3 Tables (generally available) and S3 Metadata (preview) are
available today. S3 Tables’ integration with AWS Glue Data Catalog
is in preview, allowing customers to query and visualize
data—including S3 Metadata tables—using AWS Analytics services such
as Amazon Athena, Redshift, EMR, and QuickSight.
To learn more, visit:
- S3 Tables and S3 Metadata AWS News Blog posts for details on
today’s announcements.
- S3 Tables and S3 Metadata product detail pages to learn more
about their capabilities.
- S3 Tables and S3 Metadata videos for explanations on how they
work.
About Amazon Web Services
Since 2006, Amazon Web Services has been the world’s most
comprehensive and broadly adopted cloud. AWS has been continually
expanding its services to support virtually any workload, and it
now has more than 240 fully featured services for compute, storage,
databases, networking, analytics, machine learning and artificial
intelligence (AI), Internet of Things (IoT), mobile, security,
hybrid, media, and application development, deployment, and
management from 108 Availability Zones within 34 geographic
regions, with announced plans for 18 more Availability Zones and
six more AWS Regions in Mexico, New Zealand, the Kingdom of Saudi
Arabia, Taiwan, Thailand, and the AWS European Sovereign Cloud.
Millions of customers—including the fastest-growing startups,
largest enterprises, and leading government agencies—trust AWS to
power their infrastructure, become more agile, and lower costs. To
learn more about AWS, visit aws.amazon.com.
About Amazon
Amazon is guided by four principles: customer obsession rather
than competitor focus, passion for invention, commitment to
operational excellence, and long-term thinking. Amazon strives to
be Earth’s Most Customer-Centric Company, Earth’s Best Employer,
and Earth’s Safest Place to Work. Customer reviews, 1-Click
shopping, personalized recommendations, Prime, Fulfillment by
Amazon, AWS, Kindle Direct Publishing, Kindle, Career Choice, Fire
tablets, Fire TV, Amazon Echo, Alexa, Just Walk Out technology,
Amazon Studios, and The Climate Pledge are some of the things
pioneered by Amazon. For more information, visit amazon.com/about
and follow @AmazonNews.
View source
version on businesswire.com: https://www.businesswire.com/news/home/20241203462924/en/
Amazon.com, Inc. Media Hotline Amazon-pr@amazon.com
www.amazon.com/pr
Amazon.com (NASDAQ:AMZN)
過去 株価チャート
から 11 2024 まで 12 2024
Amazon.com (NASDAQ:AMZN)
過去 株価チャート
から 12 2023 まで 12 2024
