Re:Invent 2023: More speed with serverless databases

Artificial intelligence was in the spotlight at Amazon's in-house exhibition re:invent. In order to provide the necessary computing power and speed for technologies such as the Chatbot Q, the Transcription Platform and the Titan Image Generator AWS relies on a high-performance, serverless infrastructure with the database services Aurora, ElastiCache and Redshift.

New technology for performance and scalability

The biggest innovation is the introduction of the Amazon Aurora Limitless Database. This serverless PaaS (Platform-as-a-Service) database should be able to write one million transactions per second and manage petabytes of data.

Aurora Limitless Database is based on a number of new technologies, including:

- Caspian: A specialized hypervisor that enables efficient database management on virtual machines (VM). The VMs are quickly migrated to other servers when required, which is crucial for optimizing utilization. Caspian simulates maximum main memory utilization for the VMs, while RAM resources are allocated according to demand.

- Sharding: This approach divides the database into partitions to improve scalability. A new routing system efficiently distributes requests to these partitions and combines the results, which helps to increase performance.

- Amazon Time Sync Service: This component provides precise time synchronization in the microsecond range, which is crucial for the correct processing of transactions in a partitioned database.

These technologies make the Aurora Limitless Database ideal for scenarios with high data traffic and transaction rates, such as web and e-commerce applications, AI and ML applications as well as in Industry 4.0, especially in data analysis of IoT devices. Aurora Limitless Database is currently still in the preview phase. The final version is expected to be released in 2024.

Cache without configuring

Amazon ElastiCache Serverless is designed to make it possible to create a highly available cache in the shortest possible time without having to provide or configure an infrastructure. This service uses the Caspian hypervisor to automatically adjust the size of the partitions and scale them as required.

This should speed up response times for user inquiries and significantly reduce the loading times of product pages in web, e-commerce and mobile applications.

AI optimizes data warehouses

The Redshift big data solution will not only be serverless, but will also have AI-driven scaling and optimization. When this option is activated, the system observes and learns from patterns such as simultaneous queries, their complexity and duration. Internal AWS tests have shown that this new function can achieve up to ten times better price performance for variable workloads without manual intervention.

Redshift Serverless is used for business intelligence tasks to consolidate and analyze data from different sources. It is also suitable for data warehousing to store and analyze data from different applications, as well as for processing large amounts of data in machine learning models.

Learn all about the AWS cloud with skill it

Not all of the announced innovations are available yet. But with our training courses, you are well prepared. The course provides a comprehensive overview of the AWS cloud and database services Architecting on AWS. You can refresh your knowledge of Redshift in the course Data Warehousing on AWS course.