Serverless Databases: The Good, the Bad, and the Ugly

Renato Losio is a studied engineer and certified AWS professional. Renato shared his knowledge and experience about serverless databases. First, let’s set the stage by defining “serverless.”

What Is Serverless?

Serverless means highly available without requiring the consumer to maintain the server. Serverless platforms charge per use, and they are continuously scalable. They can scale to meet fluctuations of demand. Some examples of serverless databases are DynamoDB, Azure Cosmos DB, Fauna DB, and Google Cloud Datastore.

However, most of these examples are not relational databases, which is something many organizations would like to use. Serverless relational databases are especially important for those seeking a lift-and-shift cloud migration. There are serverless databases on the market, but in order to be truly serverless, they need to overcome some current challenges. For now, we need to be aware of the caveats before deciding to switch to the existing serverless solutions.

Scaling Relational Databases to Meet Demand

Kubernetes allows for scaling, but there are limitations when it comes to relational databases. Organizations with existing relational databases might want to keep their current database structure but take advantage of serverless technology. Many relational databases tried to switch from Oracle relational databases to DynamoDB or Amazon Aurora. But this is not easy. Even Amazon, with all its resources, took years to migrate.

Problems to Overcome With Relational Databases

Connection and networking issues limit the number of application nodes that can use the data. Provisioning and scaling present limitations for the database itself, including costs and time to grow. Even infrastructure as code, although it can make things easier, does not solve the root problems.

One example of a common architecture is to use Microsoft Azure MySQL on a VM. The main advantage here is flexibility. Another is Amazon RDS for MySQL, which is a cloud-hosted version of MySQL. It has some ability to scale, and servers are managed. You can go to Amazon Aurora, which is the next level and works for an enterprise environment. But still, no scalability is built in! With Aurora, we can scale horizontally or vertically, but it isn’t quite automatic.

Automatic Scaling

There’s one solution on the Amazon AWS platform that offers automatic scaling: Aurora Serverless. It introduces automatic scaling, which is the fourth requirement for a serverless platform. The serverless version of Aurora abstracts both the scaling mechanism and the connection to the databases. It can handle unpredictable load, intermittent load, and serverless applications such as AWS Lambda.

Other platforms, such as Azure, offer similar serverless relational databases. These are becoming a ubiquitous offering from cloud providers. The goal is to be able to seamlessly switch from a standard relational database to a serverless one

Serverless Scaling Is Not Infallible

There are some issues with serverless scaling. Like all abstractions, there’s some leaking, meaning the consumer has to know a bit more about what’s under the hood beyond the interface.

Aurora, for example, is built for wide scaling and quick transactions. Long-running transactions have not performed well on Aurora Serverless. One issue is the ability to scale during long-running transactions. Is it really “serverless” like we want it to be?

Serverless databases go to sleep when they’re not being used, which means they need to wake up. This will slow down transactions until the database has had its morning coffee. We can keep it awake by using some type of ping mechanism, but this is hardly ideal.

And what about the fact that serverless databases can scale without our knowledge? Since scale implies increased cost, scaling up will run up the cloud bill. That can be a problem if there is some runaway process that causes higher-than-usual usage. We have to be aware of this and put some limitations in place.

The Bottom Line

Serverless databases aren’t just for NoSQL anymore, but we should keep in mind the limitations that exist. The abstractions are leaky, and understanding the caveats is important for successful use and especially migration. Not all applications are prepared for serverless, and some will need to be reworked to operate on this exciting, but fallible, platform.

This session was summarized by Phil Vuollet. Phil leads software engineers on the path to high levels of productivity. He writes about topics relevant to technology and business, occasionally gives talks on the same topics, and is a family man who enjoys playing soccer and board games with his children.