Why developers use Confluent to manage Apache Kafka

Imagine you are getting groceries delivered, or looking for a recommendation on what to watch next on TV, or using a credit card without worrying too much about fraud. The applications that power these interactions all rely on data in motion, and there’s a decent chance Apache Kafka powers the applications.

More than 80% of the Fortune 100 use Kafka as the event streaming substrate to power real-time, user-facing applications and software-driven back ends. Kafka has become the go-to for any organization looking to integrate increasingly diverse portfolios of applications and microservices through immutable event logs rather than mutable data stores. The benefits are manifold, but recall that Kafka is a distributed system, and volunteering to operate a distributed system yourself is an increasingly controversial choice.

This is why the cloud exists. Through fully managed cloud services, vendors bear the capital expenses and accumulate the operational expertise necessary to run infrastructure well. Confluent, the first fully managed Kafka service on the market, lets you focus on building applications and adding value to the business rather than turning dials on operationally complex infrastructure layers. I’d like to walk you through how Confluent can bring peace and simplicity to the lives of the people who work with Kafka.

Cloud-native is the future of infrastructure

There is always a greater demand for application functionality than there is the capacity to deliver it. This implies that application teams should focus on the activities that create the most value that they possibly can. Generally, this means providing new features that directly give a competitive edge to customers and users.

Of course, all applications require storage and compute infrastructure to function with ongoing development and maintenance, distracting from value-creating feature development. This is especially true for Kafka, because distributed data infrastructure imposes a significant opportunity cost on teams deciding to operate it themselves. Put simply: Your job is ultimately to take care of your customers. While running Kafka may be a means to that end, it is likely not the most practical way to get the job done. This challenge is one of many reasons that led to the rise of managed cloud services.

Elastic scaling for reals this time

Elastic scalability has always been an inherent part of the cloud’s mythology but has been slow in coming to reality. Early on in the cloud’s history, database innovators applied new approaches to horizontal elastic scalability of massive datasets. More recently, microservices and container orchestration have helped bring application scalability to the masses. However, data infrastructure generally has remained notoriously resistant to easy scalability.

Copyright © 2021 IDG Communications, Inc.

Source link