Should you use open-source databases?

You are not the only one asking this seemingly popular question! Several companies are torn between the rise in appeal of open-source databases and the undeniable challenges inherent to their adoption. Let’s explore the trends, the drivers and the challenges related to open-source database adoption.

The popularity of open-source databases

Several sources confirm the growing popularity of open-source databases. For example, the DB-Engines ranking shows that open-source databases have been overtaking commercial ones since early 2021 in terms of popularity.

According to a 2022 StackOverflow survey, 7 out of the top 10 most used databases offer open-source editions of their products:

The top 4 most used databases are, indeed, all open source.

The adoption of open-source databases is set to grow. According to this Percona survey, nearly half of IT companies are planning to increase their adoption of open-source solutions in the upcoming years:

According to the same survey, nearly 90% of the respondents already use 2 open-source databases or more.

So what’s driving the increasing popularity of open-source databases?

Drivers for open-source databases adoption

Cost savings

Moving from a commercial database system to an open-source database can lower your Total Cost of Ownership. The cost savings are due to a significant reduction in licence costs. For example, the following diagram (referenced in this article) shows a cost reduction by a factor of 3-4 between open-source databases and closed-source ones.

Avoiding vendor lock-in

Many companies do consider data as part of their most valuable assets. So, binding their future to the sole will of a single database provider is an uncomfortable situation for many.

Purchasing several closed-source database licences can help mitigate the risk of vendor lock-in. Yet, relying on open-source databases gives you additional guarantees:

• Open-source databases nurture competition. There are, for example, dozens of providers of PostgreSQL-based solutions. If one of them fails, you will find an alternative quickly with low migration costs.
• Open-source licences protect your organisation from sudden changes in provider licensing or policy. For example, OpenSearch came into existence after some companies expressed concerns about ElasticSearch’s move to the SSPL licence.

Avoiding vendor lock-in is the second most important driver (after cost savings) for adopting open source, as Percona’s findings show:

The growing maturity of open-source databases

Maturity is another important factor driving adoption. Open-source databases have been battle-tested for a few decades already. MySQL, one of the widely used open-source databases, dates back to 1995. PostgreSQL, another popular choice, dates back to 1987! So open-source databases have nothing to prove regarding successful track records.

Moreover, open-source databases have been successfully playing catch up with closed-source databases on feature sets. Oracle, for example, implemented partitioning in 1997. PostgreSQL and MySQL implemented similar functionalities in 2005 and 2008, respectively. The latter pattern is becoming more frequent as more companies contribute to open-source software.

Talent attraction and retention

Open source matters to IT professionals for several reasons ranging from better employability to embracing open-source values (e.g., collaboration, freedom).

According to the 2021 open source job report, around 97% of the surveyed hiring managers agree that “hiring open source talent is a high priority”. According to the same report, 50% of surveyed IT professionals ranked the “ability to architect solutions based on open source software … as the most valuable skill”.

Now that we have an overview of the major drivers behind the increasing popularity of open-source databases, we will discuss some of the challenges related to their adoption in the next section.

Challenges to overcome for open-source database adoption

Lack of support

According to the Percona survey (mentioned above), “lack of support” is the first concern related to open-source databases:

When it comes to supporting open-source databases, you have two main options.

The first option is to build a qualified team of database professionals that can provide support internally to the other teams in the company. The support should not only include usual DBA tasks like performance optimisation and version upgrades. It should also cover other aspects like working with the open source community to roll out/backport fixes, running scans and building releases.

This first option, albeit viable, can be challenging for many as it requires hiring and retaining highly sought-after specialised profiles. According to the 2021 open source job report, 92% of “hiring managers report difficulty finding sufficient talent with open source skills”.

This option can become even more challenging if the concerned company is planning to use several open-source databases.

The second option is to buy support services from companies providing database solutions. The purchased services might range from providing security and bug fixes to managing the whole database infrastructure on your behalf.

Lack of integrated solutions

To successfully run a reliable database deployment, you don’t only need a database engine. You need to test, use and maintain several additional tools and extensions to provide high availability, monitoring, alerting and backup for your installation.

Closed-source databases, like Oracle database, provide these mentioned functionalities with tools that are shipped within the database engine (or tightly integrated with it). Most of these tools are covered by the purchased vendor support.

With open-source databases, like PostgreSQL, you need to test a few options (think of repmgr, Patroni, Stolon, PAF) with the rest of your set-up to ensure that they meet your requirements.

Basically, with closed-source databases you have less choices but you have an integrated solution that covers most of your needs. With open-source databases, you need to have the expertise and time to select, test and maintain the additional tools that you will need to cover your requirements.

Migration complexity

Migrating from an already used database to another solution often needs careful planning and testing. This is true for any migration between different database engines (open-source or not).

Even among the same family of databases (e.g., relational databases, document databases), you might need – for example – to re-write some of your queries to avoid performance degradation. Every engine implementation is unique and, therefore, will excel in some cases and underperform in others.

Deciding on a migration strategy depends on several factors. Here is a list of some of the important ones:

• Database size; the larger your databases, the more complex the migration can get
• The outage constraints related to your business: how long of an outage can you afford per migration?
• The dependencies between your applications and their databases. Can you migrate every database independently, or do you need to migrate them in batches of related databases?
• The available infrastructure for the migration: do you need to re-use the existing infrastructure? Are you planning to combine the database migration with a change in the infrastructure?

To summarise, lack of support, poor integration and migration complexity are some of the challenges organisations face when adopting an open-source database. Canonical offers a variety of solutions for organisations looking to overcome these challenges and get the most out of open-source in this space.

Canonical solutions for open-source databases

Expanded support and regulated environments

Canonical offers, through Ubuntu Pro and Ubuntu Advantage, up to 10 years of fixes for high and critical CVEs not only to Ubuntu itself but also to around 25,000 accompanying deb packages. The covered deb packages include popular open-source databases like MongoDB, Redis and PostgreSQL.

Moreover, Canonical helps organisations comply with a wide range of certifications and standards like DISA-STIG, FedRAMP and ISO27K.

Operators for open-source databases

Canonical provides a growing list of Juju-based operators to manage the lifecycle of your database clusters. 

Canonical’s database solutions are a suite of integrated open-source tools. They provide all the needed functionalities to deploy, scale , backup, monitor and maintain your database deployments.

Our Juju operators go through a growing list of unit and integration tests to ensure high quality for the whole solution. 

Moreover, our operators work with bare-metal, virtual machines and Kubernetes. Juju can deploy your databases on the major cloud providers (e.g., AWS, Azure, GCP, Oracle). So, it will be up to you to decide what fits you best!

Canonical support

We are already using open-source databases in hundreds of our products deployments. We gained extensive knowledge operating them in different environments. Our field engineering teams can help you:

• Plan the migration of your data to the database of your choice
• Design a database solution that meets your requirements
• Manage the infrastructure for you

Contact us to speed up open-source database adoption in a scalable and secure way.