The latest release (2.7.0-b190) of Cloudera DataFlow (CDF) on CDP Public Cloud introduces new Change Data Capture processors, flow version tagging, deployment configuration reuse, NiFi bulletin monitoring and supports new Kubernetes versions.   Key Deployment & Platform features for this release Latest NiFi version Flow Deployments and Test Sessions now support the latest Apache NiFi 1.24 release. New Debezium CDC processors Customers can now build and deploy flows with Debezium based CDC processors for MySQL, Postgres, Oracle, SQLServer and DB2 databases. Deployment alerts for NiFi bulletins CDF now detects bulletin error messages in NiFi flow deployments and displays them as a warning in the dashboard. Flow Definition versions can now be tagged Customers can now use tags to identify versions easier. Common use cases for tags are applying a custom versioning scheme, or labeling flow versions as ‘ready for deployment’. The CLI introduces new commands to search flow definitions for tags and streamlines the CI/CD process. Deployment configurations can now be exported and reused Customers can now export a configuration archive of existing deployments and reuse them when creating new deployments. This speeds up redeploying new versions of the same flow through the Deployment Wizard by filling in parameter values, KPI configurations etc. from the deployment configuration.  Support for new Kubernetes versions AKS 1.27 / EKS 1.27 New ReadyFlows for Debezium CDC to Kudu (Tech Preview) S3/ADLS to Delta Lake HuggingFace dataset to S3/ADLS S3 to Watson.x   To learn more about this release, see the product documentation,  take the interactive Flow Designer product tour and visit the product page! ... View more

The latest release (2.6.1-b92) of Cloudera DataFlow (CDF) on CDP Public Cloud introduces fine-grained access controls for flow drafts and deployments within an environment and support for creating and managing reporting tasks through the CLI. Key Deployment & Platform features for this release Fine-grained access control for drafts and deployments Create projects, assign drafts and deployments, add CDP users/groups to control access. Reporting Task support Monitor your deployments with NiFi’s out of the box or custom Reporting Tasks! Reporting Tasks can now be added to deployments using new CLI commands cdp dfworkload create-reporting-task  Key Flow Designer features for this release Additional Zoom control Users without a scroll wheel or trackpad can now zoom in/out using ‘+/-’ controls on the canvas Collapsible sections in configuration drawer Settings, Scheduling, Properties and Relationships sections can now be collapsed for more efficient flow design   To learn more about this release, see the product documentation, take the interactive Flow Designer product tour and visit the product page! ... View more

The latest release (2.6.0-b311) of Cloudera DataFlow (CDF) on CDP Public Cloud introduces NiFi 1.23, vertical scaling, several cost optimization features, developer productivity improvements and the ability to collect Diagnostic Bundles through the UI. Key Deployment & Platform features for this release Latest NiFi version: Flow Deployments and Test Sessions now support the latest Apache NiFi 1.23 release. Up to 10% savings on compute resources by removing Zookeeper from deployments and test sessions. CDF now uses a NiFi and Kubernetes native clustering protocol. Up to 50% savings on storage cost (AWS only) by moving from GP2 to GP3 volumes and introducing a new, cost efficient ‘standard’ storage profile New ‘Suspend Deployment’ behavior now terminates associated compute resources resulting in Cloudera and infrastructure cost savings. Manual Vertical scaling allows customers to change the deployment sizes easily after a deployment has been created. Allows customers to start small and easily scale to bigger nodes as needed. Select between ‘standard’ and ‘performance’ storage profiles when creating a deployment based on your needs. Create Diagnostic bundles and attach to support cases for faster support case resolution. Allows customers to easily collect relevant logs without sifting through S3 or Kubernetes pods. The Dashboard now displays alerts for failed Kubernetes resources like NiFi pods, statefulsets etc. This makes it easier to see when there are underlying issues on the K8s cluster. A new ‘workspace view’ for centralized resource management allows users to see all resources (deployments, drafts, inbound connections, custom NAR configs) in a central place. Support for new Kubernetes versions AKS 1.26 / EKS 1.25 Key Flow Designer features for this release Support for sensitive dynamic properties The Flow Designer now supports creation of sensitive dynamic properties for several components like InvokeHTTP and ExecuteStreamCommand Download NiFi app log directly from the test session for easier troubleshooting when access to logs is needed during development  Copy bulletin messages using a ‘copy’ icon to share them with colleagues or support To learn more about this release, see the product documentation, take the interactive Flow Designer product tour and visit the product page! ... View more

The latest release (2.5.0-b210) of Cloudera DataFlow (CDF) on CDP Public Cloud introduces NiFi 1.21, flow metrics driven auto-scaling, advanced UIs for JoltTransformJSON and UpdateAttribute processors, bulk actions in the Flow Designer, UDR support on Azure and makes in-place upgrades generally available to all customers.   Flow Deployments and Test Sessions now support the latest Apache NiFi 1.21 release. In-place upgrades are now generally available to all customers. For more information, see Service upgrade. Flow Deployments now support Flow Metrics Scaling in addition to CPU utilization based auto-scaling. For more information, see Auto-scaling flow deployments When selecting Private Cluster during enablement on Azure, CDF no longer provisions a public load balancer and now supports User Defined Routes (UDR). Users can now provide their own values for the Kubernetes Pod CIDR Range and Kubernetes Service CIDR Range during enablement CDF now inherits load balancer configuration including available subnets from its associated CDP environment during the enablement process New Flow Designer Features The Flow Designer now supports multi-selection on the canvas and bulk actions for Start, Stop, Enable, Disable, Move, Change parent group, Copy/Paste, and Delete. The Flow Designer now supports the advanced configuration UI for UpdateAttribute. The Flow Designer now supports the advanced configuration UI for JoltTransformJson.   The Flow Designer now supports Birdseye and Zoom controls.   The Flow Designer now supports Processor Diagnostics with an active Test Session. New ReadyFlows The following new ReadyFlows have been added to the ReadyFlow Gallery: CDW Ingest CDP Kafka to Snowflake Slack to S3 Updated Confluent Cloud to Snowflake using new Snowpipe processors To learn more about this release, see the product documentation, take the interactive Flow Designer product tour and visit the product page! ... View more

The latest release (2.4.0--b317) of Cloudera DataFlow (CDF) on CDP Public Cloud makes the Flow Designer generally available to all CDP Public Cloud customers, introduces a streamlined way to collect log files, supports new NiFi and Kubernetes versions and introduces several usability improvements when creating and managing flow deployments through the UI.   The ability to design flows with the new Flow Designer is now generally available to all CDP Public Cloud customers. The CDP cli now supports commands to generate diagnostic bundles for CDF for troubleshooting Flow Deployments and Test Sessions now support the latest Apache NiFi 1.20 release When creating a deployment through the deployment wizard, users can now filter for sensitive or empty parameters CDF now creates Kubernetes clusters with version 1.23 in AWS EKS and 1.24 in Azure AKS. Upgrading and restarting deployments now triggers a NiFi Health Alert if the NiFi cluster does not return to a healthy state. This can be cleared by performing a Restart Deployment once the underlying issue has been fixed. Restart Deployment now supports force suspending the NiFi Flow in case the running flow itself is impairing the health of the NiFi cluster. Enable DataFlow action now performs validation steps immediately and fails fast if there are issues with the request Some enablement failure messages are improved when the failure is due to common issues surrounding volume provisioning The following ReadyFlows have been added to the ReadyFlow Gallery: HubSpot to S3/ADLS JDBC to JDBC Kafka to Apache Iceberg Shopify to S3/ADLS To learn more about this release, take the interactive Flow Designer product tour , see the product documentation, and visit the product page! ... View more

The latest release (2.3.0-b347) of Cloudera DataFlow (CDF) on CDP Public Cloud introduces the following new features for both, AWS and Azure customers:   Flow Designer [Technical Preview] Developers can now build new data flows from scratch using the integrated Designer. Flow Drafts can be tested using Test Sessions before they are published to the Catalog. Once they are in the catalog, users can create Flow Deployments using the Deployment Wizard or CLI. Note: The Flow Designer is a Technical Preview feature and requires an entitlement to be activated. Reach out to your Cloudera Account team to get Technical Preview access.   Added support for certificate renewal of inbound connection endpoints Six new ReadyFlows have been added to the ReadyFlow Gallery: Airtable to S3/ADLS Box to S3/ADLS Dropbox to S3/ADLS Google Drive to S3/ADLS MQTT to Kafka Salesforce to S3/ADLS DataFlow is now also available in the EU and APAC regional CDP control planes. Updated to support using kubernetes 1.23 EKS and AKS clusters. General improvements to DataFlow upgrade reliability to allow for improved success rates Includes new NiFi version 1.18.0.2.3.7.0 for flow development and flow deployment. Updated DF Function binaries to use 1.18.0 See the Flow Designer Announcement blog post and the product documentation for more information. ... View more

The latest release (2.2.0-b194) of Cloudera DataFlow (CDF) on CDP Public Cloud introduces the following new features for both, AWS and Azure customers:   A streamlined first time user experience You can use the new Hello World ReadyFlow to create your first flow deployment without dependencies on source or target systems. New landing pages for the Dashboard and Catalog allow you to get started with just one click. Four additional new ReadyFlows are available in the ReadyFlow Gallery Non-CDP ADLS to S3/ADLS Non-CDP S3 to S3/ADLS ListenTCP filter to S3/ADLS ListenSyslog filter to S3/ADLS Endpoint Hostnames certificates can now be renewed when using Inbound Connections Learn more about the new features in the CDF-PC documentation and take an interactive tour of CDF-PC. ... View more

The latest release (2.1.0-b123) of Cloudera DataFlow (CDF) on CDP Public Cloud introduces the following new features for both, AWS and Azure customers:   [Technical Preview] CDF service and deployment upgrades are now supported. CDF service upgrades require a specific entitlement that is granted by request only. Reach out to your Cloudera Account Representative or open a support ticket to request CDF service upgrades. The create-deployment CLI command now supports Inbound Connections. CDF now correctly configures the deployment when it detects any of the two reserved Controller Services Default NiFi SSL Context Service and Inbound SSL Context Service regardless of whether they are external references or not. Kubernetes Version 1.22 is now the default Kubernetes version for CDF. When CDF is enabled, it creates AWS EKS and Azure AKS clusters based on version 1.22. CDF now supports Data Lake scaling. After the Data Lake scale operation has completed, affected deployments will show an active alert and need to be manually restarted to put the new configuration in effect. The Event Hub to ADLS ReadyFlow now allows you to specify an Event Hub Service Bus Endpoint.   Learn more about the new features in the CDF-PC documentation and take an interactive tour of CDF-PC. ... View more

This release (CDF-PC 2.0.0-b302) introduces the following new features for both, AWS and Azure customers:   Flow Deployments can now be configured with Inbound Connections providing a stable hostname, automated certificate/key generation as well as automated load balancer provisioning to enable client applications to securely send data to a NiFi Flow Deployment. Flow Deployments now support the latest Apache NiFi 1.16 release. Flow Deployments on Azure now use Premium SSD disks to store NiFi’s various repositories. Python 3.9 is now installed on NiFi containers allowing users to run python scripts through NiFi processors. Requests and urllib3 libraries are also installed by default. Information level alerts are now displayed in the Dashboard to inform users about an action they should perform for a Flow Deployment without changing the Flow Deployment status to “Concerning”. Deployments can now be restarted from the Deployment Manager to perform actions that require a NiFi process restart. Filters that have been set in the Dashboard are now persisted after a user has navigated to the Deployment Manager and returned back to the Dashboard. When enabling DataFlow, you can now specify CIDRs as a comma separated list to configure Load Balancer and Kubernetes API Server Endpoint Access. CDF-PC integration into the CDP CLI is now generally available. Make sure you run the latest CDP CLI version to use cdp df commands. Note: When release 2.0.0-b302 becomes generally available, the CDP CLI version supporting cdp df commands may not be available yet. In this case, use the beta CDP CLI. The following ReadyFlows have been added to the ReadyFlow Gallery: JDBC to S3/ADLS Non-CDP ADLS to CDP ADLS Non-CDP ADLS to CDP ADLS Azure Event Hub to ADLS ListenHTTP filter to Kafka Confluent Cloud to S3/ADLS Confluent Cloud to Snowflake Learn more about the new features in the CDF-PC documentation and take an interactive tour of CDF-PC. ... View more