What's New @ Cloudera

Find the latest Cloudera product news
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Apache Flink is Now Generally Available in the Cloudera Streaming - Kubernetes Operator

avatar
Super Guru

We are excited to announce the release of the Cloudera Streaming Analytics Operator, as an integral component of Cloudera Streaming - Kubernetes Operator. The Cloudera Streaming Analytics Operator brings Apache Flink and Cloudera SQL Stream Builder to Kubernetes for our customers running Cloudera Streaming - Kubernetes Operator. This is a key release that adds strong streaming analytics capabilities to complement the Kafka features of the Operator. Cloudera SQL Stream Builder is available in Tech Preview in this release.

While Kubernetes started out as a container orchestration platform where customers would mostly run stateless applications and APIs, it has now matured to a point where it can run stateful, large scale data processing frameworks such as Apache Flink and Apache Kafka. We have seen this shift in Kubernetes usage in our customer base and are excited to meet them where they are. Enabling Flink applications on Kubernetes is an important part of our vision and complements Cloudera's end-to-end Data in Motion story on Kubernetes.

araujo_0-1721312235888.png


Use Case

The Cloudera Streaming - Kubernetes Operator allows Flink and Kafka customers to deploy their workloads on existing shared Kubernetes clusters. The Kubernetes Operator automates the installation and lifecycle management of applications on Kubernetes, significantly lowering the cost of operations and allowing customers to focus on use case implementation instead of cluster operations.

For customers, this means that they can take advantage of:

  • Greater hybrid portability and accelerated deployments via streamlined life cycle management
  • Increased resource efficiency via shared clusters
  • Improved scalability via Kubernetes orchestration tools like OpenShift


Release Highlights

  • Cloudera Streaming Analytics Operator 1.0 (Flink and SQL Stream Builder)
    • Deploys and manages Flink and SQL Stream Builder
    • Supports Flink 1.18.1
    • SQL Stream Builder is available in Tech Preview (no support)
    • Auto-scaling for Flink deployments
    • PyFlink support
    • Enterprise ready
    • Prometheus integration for monitoring

This adds to the the following Streaming Operator capabilities, released earlier:

  • Cloudera Streaming Messaging Operator 1.0 (Kafka)
    • Deploys and manages Kafka, Zookeeper, and Cruise Control
    • Supports Kafka 3.7
    • Provides rack awareness and high availability
    • Supports various authentication options (OAuth, LDAP, mTLS, PLAIN)
    • Supports authorization through Kafka Access Control Lists (ACL)
    • Prometheus integration for monitoring

Learn More

  • Watch the recording of our release webinar showcasing an end-to-end financial services use case running on the new Kubernetes Operators.
  • Read the Operator documentation for detailed information about pre-requisites, installation and configuration