We are excited to announce the General Availability of Cloudera Flow Management - Kubernetes Operator and Cloudera Streaming - Kubernetes Operator, allowing Cloudera customers to run Apache NiFi and Apache Kafka on Kubernetes.
While Kubernetes started out as a container orchestration platform where customers would mostly run stateless applications and APIs, it has now matured to a point where it can run stateful, large scale data processing frameworks such as Apache NiFi and Apache Kafka. We have seen this shift in Kubernetes usage in our customer base and are excited to meet them where they are.
Use Case
The Kubernetes Operators allow NiFi and Kafka customers to deploy their workloads on existing shared Kubernetes clusters. Kubernetes Operators automate the installation and lifecycle management of applications on Kubernetes, which significantly lowers cost of operations and allows customers to focus on use case implementation instead of cluster operations.
For customers, this means that they can take advantage of:
- Greater hybrid portability and accelerated deployments via streamlined life cycle management
- Increased resource efficiency via shared clusters
- Improved scalability via Kubernetes orchestration tools like OpenShift
Release Highlights
- Flow Management Operator GA (NiFi):
- Deploys and manages NiFi and NiFi Registry
- Supports NiFi 1.25 and offers NiFi 2.0 (Tech Preview)
- Automatically creates zookeeper-less NiFi deployments
- Provides cfmctl command line tool for easy installation & setup
- Manual and (CPU based) auto-scaling of NiFi clusters
- Increased durability and availability through auto-recovering NiFi pods
- Streaming Operator GA (Kafka)
- Deploys and manages Kafka, Zookeeper, and Cruise Control
- Supports Kafka 3.7
- Provides rack awareness and HA
- Supports various authentication options (OAuth, LDAP, mTLS, PLAIN)
- Supports authorization through Kafka ACLs
- Prometheus integration for monitoring
Learn More
- Join us for our release webinar showcasing an end-to-end financial services use case running on the new Kubernetes Operators.
- Read the Operator documentation for detailed information about pre-requisites, installation and configuration
