The Data In Motion Team is pleased to announce the release of Cloudera Flow Management Operator for Kubernetes version 3.3. This release focuses on improving security, streamlining deployments, and enhancing upgrade paths. The core highlights include first-class Apache Ranger authorization, seamless inter-cluster communication, declarative flow deployments, and rolling upgrades for NiFi 2 clusters. Release Highlights: Unified Security Governance Across NiFi Clusters: Centralized, policy-based access control for Operator-Managed NiFi clusters is now possible by integrating customer-managed Apache Ranger with the Cloudera Flow Management Operator for Kubernetes.  Automated, Secure Cross-Cluster Communication: The Operator now automatically provisions SSL context services using existing node keystores and truststores, eliminating manual TLS configuration for secure cluster-to-cluster data movement. Zero-Downtime Rolling Upgrades for NiFi 2: Upgrade between NiFi 2 versions smoothly without service interruption, keeping at least one node running throughout the upgrade process to maintain continuous availability. Declarative Flow Deployments: Treat flows as code by importing and automatically starting NiFi flow definitions directly from Kubernetes ConfigMaps or Secrets via a new ProcessGroup Custom Resource. Please be aware that at v1: Deployed flows cannot be updated. Changes to the source ConfigMap or Secret are not applied after deployment. Parameters must be defined within the Flow Definition JSON. External parameter contexts are not supported. Automated Kubernetes Resource Cleanup: Reduce cluster clutter and administrative overhead by automatically identifying and purging obsolete resources after upgrades or during configuration changes. Upgrading to the New Release: Reference the latest operator version in the Helm Install command. More details can be found in the installation instructions.  Helpful Links: Release notes Documentation ... View more

The Data In Motion Team is pleased to announce the release of Cloudera Data Flow 3.1 for Cloudera on cloud. In this release, we’ve focused on removing some friction that slows teams down. By introducing command line interface (CLI) automations and advanced debugging tools, we’re giving developers more visibility and admins more control. These efficiency gains are backed by a leaner, more performant Flow Designer and a hardened foundation that stays ahead of evolving security standards. Release Highlights: Command-Line Test Session Management: Users can now manage Flow Designer test sessions via the CLI. This update enables seamless automation of test session  workflows, eliminating repetitive manual steps in the UI. Streamlined Parameter Orchestration: We’ve extended full CLI support to Shared Parameter Groups, including the ability to reference them by CRN. By removing the need for exhaustive group details during CLI deployment and enabling full Create, Read, Update, Delete (CRUD) operations via the CLI, we’ve simplified your Continuous Integration/Continuous Delivery (CI/CD) pipelines and established a secure and centralized source of truth for sensitive configurations. Automate Workspace Cleanup: Keeping your development environment organized is significantly faster. Users can now list and delete unused flow drafts directly via the CLI, replacing manual, one-by-one user interface (UI) deletions with automated commands. Native Sparkplug IoT Support: Customers managing industrial data can now ingest Sparkplug-compliant streams natively. The new ConsumeMQTTIIoT processor and MQTTIIoTReader remove the requirement for custom NiFi Archive (NAR) files or external Python processors. This streamlines Sparkplug IoT architectures and eliminates the overhead of managing custom components in production. A Faster, More Efficient Flow Designer: Optimized Performance: Component referencing now loads in a fraction of a second—down from over a minute in large flows—ensuring a fluid experience even in high-concurrency environments. Reduced Overhead: Overall central processing unit (CPU) usage for canvas event processing has been reduced by 60–80%, resulting in a snappier, more cost-effective canvas. Enhanced Visibility: We have extended full Data Provenance support to NiFi 1.x flow drafts, giving you the deep diagnostics to troubleshoot drafts with the same precision as NiFi 2.x drafts. ReadyFlow Updates:  Kafka to Snowflake and Confluent Cloud to Snowflake ReadyFlows updated to use key-pair authentication (StandardPrivateKeyService) to comply with Snowflake's mandatory multi-factor authentication requirement. S3 to IBM watsonx ReadyFlow updated to enable choice of LLM model, from available options, versus forcing a default. Platform and Security Updates: Seamless Ingress Migration: We’ve transitioned to Traefik as our cluster ingress controller, delivering more robust routing and enhanced Transport Layer Security (TLS) handling. To ensure zero friction, this migration is performed automatically during a standard upgrade process. Hardened Security Posture: To stay ahead of evolving threats, we’ve replaced Valkey and cert-manager base images with Chainguard equivalents. This significantly reduces the attack surface of your Cloudera Data Flow environment without requiring any changes to your flows. Proactive Cost Governance: New alerts for unused Inbound Connections help you maintain a lean environment. By identifying gateways no longer in use, you can proactively eliminate unnecessary cloud costs and minimize your security footprint. Up-to-date Infrastructure and Runtime: Cloudera Data Flow 3.1 introduces official support for Kubernetes 1.34 and provides the latest security and stability hot fixes for NiFi 1.28.1 and 2.6.0, ensuring your environment remains current and compliant. Upgrading to the New Release: Customers can perform in-place upgrades from supported Cloudera Data Flow versions to 3.1. Alternatively, disabling and re-enabling an existing Data Flow environment will result in the re-enabled environment running the latest version. Note: Before initiating the upgrade, please ensure that you have reviewed the NiFi EoS requirements and upgrade prerequisites.  Important Advisory for Cloudera Data Flow on Azure: To maintain a seamless, standard upgrade experience, we recommend upgrading to Cloudera Data Flow 3.1 by July 31, 2026. This timeline ensures your upgrade path is as straightforward as possible. Helpful Links: Release notes  Documentation ... View more