Cloudera DataFlow 2.9 now supports building GenAI pipelines with NiFi 2, simplifies parameter sharing and improves monitoring capabilities

[#1]

This release (2.9.0-b383) of Cloudera DataFlow on CDP Public Cloud increases developer productivity through the introduction of Parameter Groups which can be shared between flow drafts. Developers can now also create NiFi 2.0 flows in the Designer leveraging new Cloudera exclusive processors for building RAG data pipelines. Deployments can now be configured with a Prometheus endpoint that allows scraping NiFi metrics. DataFlow’s service and deployment events and alerts now support in-app and email notifications. 

Demo Video of release highlights

Key features for this release

• Develop NiFi 2 flows using the Flow Designer (TP):
  Customers can now try out NiFi 2 easily in the Flow Designer and start building flows without having to set up new, dedicated clusters. 
  
  
  
• New Cloudera exclusive GenAI processors and ReadyFlows (TP):
  Customers can now implement RAG pipelines by using new processors to parse, chunk and vectorize data, bringing context to their LLMs.
  
  
  
• Shared Parameter Groups:
  Developers can now centrally define and manage parameters and re-use them for multiple drafts, eliminating tedious copy & paste of parameters and their values.
  
  
  
• New ‘Resources’ page:
  Users can now easily view and manage all their workspace resources like deployments, drafts, parameter groups, inbound connections, custom NAR/Python configurations in a single place.
  
  
  
• Notifications for DataFlow events:
  Administrators can now configure notifications for DataFlow service and deployment level events. Users can subscribe to be notified via the Cloudera Management Console or Email to be always up to date about the health of their service and deployments.
  
  
  
• NiFi metrics Prometheus endpoint:
  Administrators can now configure deployments to expose NiFi metrics through a Prometheus endpoint. Customers can configure their Prometheus instances to scrape these endpoints, consume relevant metrics and build custom dashboards on top.
  
  
  
• Latest NiFi versions:
  Test sessions and deployments now support NiFi 1.27 and NiFi 2.0 (TP) versions.
• New Kubernetes version support:
  Cloudera DataFlow now supports EKS/AKS 1.29

To learn more about this release, see the product documentation,  take the interactive Flow Designer product tour and visit the product page!