Ask the Community

Good morning to the community, I would like to raise a problem that I am having with a nifi cluster that I manage version 1.11.4, during the night a nifi node goes down, every day for about a month now, when we were developing some contingency processes it started to fail with the fall of a node, although it is true that I am clear about which processes it is, I need to understand how I can repair the problem from the process front and from the nifi front, attached is the nifi app_user.log log and the zookeeper log, to see if you can help me.log user_app.loglog zookeeperI look forward to any support. Greetings to all.

Hello community, greetings again. I'm currently analyzing how to migrate from NIFI 1.11.4 to 2.0. My strategy is to create and export my workflow template to version 1.9.2, which worked correctly. Now, from there, I want to migrate to version 2.0. Using the same method, all of this done locally on Windows, to later install the final version of the NiFi cluster from a Docker image, the problem I encountered is that version 2.0 of NiFi requires a username and password. Upon reviewing comments and documentation on the matter, I see that they indicate extracting the user from the nifi-app.log file, searching for "0" in the "Generated Username" file, but that doesn't appear in my log file, and I also can't find the Windows command to generate the user; I only see it for Linux. My question is, is there any method to generate users in Windows, or is there any... HOW TO DEFINE IT IN THE login-identity-providers.xml FILE?

Hello,We have a 3 nodes cluster. We migrate our system from Nifi 1.23.2 to Nifi 2.4 with a stop at Nifi 1.28.1 as recommended in migration docs.Our flows were working as expected for the part 1.23.2 -> 1.28.1. We prepared the migration to Nifi 2 as explain in docs (remove variables and deprecated processors, modified CRON to remove the 7th digit...). Tests were OK.We then upgrade from 1.28.2 to Nifi 2.4. Just after this upgrade, our flows containing queues with round robin strategy getting stucked.One example, we ingest a XML file in the primary node (let's call it node0), do some stuffs, no problem. We then split the file by lines and send each lines to an EvaluateXPath to analyse the data. This queue between the SplitXML and EvaluateXPath has a round robin strategy to improve perfs. Just after the upgrade from 1.28.1 to Nifi 2.4 only one third of the lines were processed the lines that should be sent to the node1 et node2 are not processed and stay in the queue. Only reccords

When I try to run a query against the nifi 2.5.0 API to get info about provenance activities, it seems to be quite difficult.I installed NIfi on my PC with no modifications and all default settings, and just use the default user and password that's created when you install it.I run this in my local windows CMD window and get   "unknown error" :curl --insecure -X DELETE -H "Authorization: Bearer <my token> " https://localhost:8443/nifi-api/provenance/queriesThen I run this and get : curl --insecure -X DELETE -H "Authorization: Bearer <token>" https://localhost:8443/nifi-api/provenance/queries > test.txtIn text.txt file I get :"An unexpected error has occurred. Please check the logs for additional details."I'm starting to think the Provenance part of the API might be broken as it shouldn't be this hard?I ran other API queries and they work fine.What I'm trying to do is get a list of all files that have passed through my small local nifi installation so we ca

We need to set up OIDC authentication in Apache NiFi. However, our OIDC provider is only reachable through an HTTP/HTTPS proxy. How can we configure NiFi to use this proxy when performing OIDC authentication? Are there specific properties that must be added to nifi.properties or any additional configuration required so that NiFi accesses the OIDC provider through the proxy?Thank you for your response 

 Hello,For a project, I need to configure a WebSocket connection using NiFi. I also need to define a custom sub-protocol in the “Sec-WebSocket-Protocol” header.I have seen that some custom headers can be defined in the parent FF with the “header.Something” attribute. In fact, all headers are defined except for the sub-protocol. How can I achieve this without having to develop my own processor?My GenerateFlowFile has the following attributes: `header.Sec-WebSocket-Protocol = Test` and another `header.Test = Test2`.The request retrieved by Wireshark only uses header.Test. Is it normal ?Thanks.(related to https://community.cloudera.com/t5/Support-Questions/Nifi-websocket-subprotocols/m-p/311764#M224816 and https://community.cloudera.com/t5/Support-Questions/NiFi-WebSocket-Headers/m-p/361488)

When trying to login via sso I receive this error message.  403There was a problem authenticating your request.Cloudera SSO is disabled for Account '14f772e1-c5b2-48f5-b354-993f95d0da10'. To log in, use your identity provider. To re-enable Cloudera SSO Login, speak with your CDP administrator.Request ID: db73097d-1e99-4b98-a342-feff906ba5aa

I need to pull data from pager duty that needs a date in the url.  This is an example:https://api.pagerduty.com/oncalls?include%5B%5D=users&schedule_ids%5B%5D=AAAAAA&schedule_ids%5B%5D=BBBBBB&schedule_ids%5B%5D=CCCCCC&schedule_ids%5B%5D=DDDDDD&since=2025-12-04&until=2025-12-05&earliest=trueThe two parameters are since and until.  The intent is this runs once a day and a new date value set for each day.  Something tells me it may look like this:https://api.pagerduty.com/oncalls?include%5B%5D=users&schedule_ids%5B%5D=AAAAAA&schedule_ids%5B%5D=BBBBBB&schedule_ids%5B%5D=CCCCCC&schedule_ids%5B%5D=DDDDDD&since=#{since_date}&until=#{until_date}&earliest=true but the how I don't know.  I'll continue searching, but if anyone has a solution I will be thankful.

Dear TeamI am currently pursuing the CLOUDERA AI Technical Professional Accreditation (CAITP) certification. I have completed the first two courses successfully; however, I am unable to move forward to the third course because there is no “Launch” button available.Additionally, although the first two courses show as completed individually, the overall certification progress is not updated and does not reflect the completion percentage. Due to this issue, the certification appears to be blocked, and I am unable to continue with the remaining courses.For comparison, I have already completed the CLOUDERA AI Sales Professional Accreditation (CAISP) certification without encountering any issues.Could you please assist me in resolving this problem and enable access to the remaining CAITP courses?Thank you, and I look forward to your support.

Dears,my cluster is TLS and Kerberized, and when try to add CFM 2.1.7 ( NIFI ).i used https://docs-archive.cloudera.com/cfm/2.1.1/cfm-security/topics/cfm-security-ldap-authentication.htmlto configure LDAP however, i go the below error from NIFI WebUI "Authentication Proxy Server not trusted"Any idea how to solve this.

Hello,In Cloudera Data Flow, how should an InvokeHTTP processor be configured to be able to invoke the underlying nifi's rest api? I've tried several (localhost, with and without ssl etc.) options, but the connection is always refused.Thanks,Peter

I have nifi cluster hosted on AKS , its 3 node  with memory 20G each node. node has dedicated pvcs standard lrs.We are seeing occasionally once in 1/2 weeks nifi node-0 only disconnects  with below error.We are running full load ingestions frequently. but there is no pattern . we  run 3-4 processor groups  have overlapping time (parallel).  but not all the time we get error . most of the runs on most of the day work . sometime we get below mentioned error.   what we observed . node-0 first gets disconnected later when it tried to reconnect  org.apache.nifi.controller.serialization.FlowSynchronizationException.Any help possible ? Error dump : 2025-11-01 06:05:19,513 WARN [Notify Cluster of Node Status Change-2] o.a.n.c.p.i.StandardClusterCoordinationProtocolSender Failed to send Node Status Change message to nifi-0.nifi-headless.nifi-highresource-lts.svc.cluster.local:8443org.apache.nifi.cluster.protocol.ProtocolException: Failed to

Hi Team,I have a usecase of JVM application(Java) to be build and deployed on CML.  In  Cloudera data platform,  I see option of DE, Data warehouse, Cloudera AI, Data catalog,  etc. Even after exploring all the option, I could not figure out a straight forward way for the same. Please guide.        

Hi,We have deployed Cloudera Flow Management through the Kubernetes Operator on a k8s cluster. What's the best practice how to add custom processor NARs in this environment?On a plain (not containerized) NiFi deployment I would put the NAR into the extensions sub folder of the NiFi deployment and it gets unpacked during startup of NiFi.How does it work with the Cloudera Kubernetes Operator version of NiFi? We have a NFS share available that could be mapped to a container's sub folder.Has anyone done this successfully?Thanks,-karsten

I can able to connect the impala using isql when I use to connect the same using pyodbc I am getting this error Error: ('H000', '[H000] [ (50404) (SQLDriverConnect)')

Hi  , I want to automate my NiFi flow deployment process.Currently, I have a NiFi template generator that works with multiple XML and configuration files where I define properties like Kafka topic names, broker IPs, and other NiFi-related settings. Using this template generator, I parse an existing sample NiFi XML template and generate a new XML file with all the required properties populated.Right now, I upload this generated XML file manually through the NiFi UI. After that, I manually create a new process group on the NiFi canvas, import the template into this group, and then start the processor group and all required controller services.I want to automate this entire flow—right from generating the template and creating the process group, to uploading the template into NiFi and starting the processors and controller services—so that no manual steps are needed in the NiFi UI.

HiWhat is the preferred method to alter a nifi processor configuration please?We want to be able to get a user to choose from one of 3 options from a drop down menu in a GUI screen. The chosen option then is passed to either a curl command to make changes to the processor via the Nifi API, or can we create a python command to then alter a nifi processor configuration possibly by curl, but maybe there is another way?Curl looks a safer option, but python might be better? Not sure....?Thanks in advance. 

Hi everyone, I hope you’re doing well. I am working on a dataflow in Apache NiFi 1.18, and I need to retrieve the queue size information (flowfile count and content size) directly within NiFi itself, not via an external script or Postman. I know that the NiFi REST API provides this data, and I can access it successfully using external tools. However, my goal is to access queue metrics from inside NiFi, for example through processors like InvokeScriptedProcessor, QueryNiFiReportingTask, or any other built-in mechanism, without sending an external REST API request from outside NiFi. Is there a recommended approach, processor, or reporting task that allows NiFi to read its own queue sizes internally? If not, what would be the best practice to achieve this? Any guidance or examples would be greatly appreciated. Thank you in advance!

Hi everyone,I installed Spark cluster standalone. I want to config send log job to minio. I configured spark-default.conf file as bellow.  spark.eventLog.enabled=false spark.eventLog.dir=s3a://spark-logs/ spark.history.fs.logDirectory=s3a://spark-logs/ spark.hadoop.fs.s3a.endpoint=http://192.168.182.131:9000 spark.hadoop.fs.s3a.access.key=admin spark.hadoop.fs.s3a.secret.key=admin12345 spark.hadoop.fs.s3a.path.style.access=true"  When I submit job spark-submit --master spark://spark-master-svc:7077 --conf spark.jars.ivy=/tmp/.ivy2 pi.py 1. I got the error:5/10/26 12:13:29 INFO SparkContext: Successfully stopped SparkContextTraceback (most recent call last): File "/opt/bitnami/spark/examples/src/main/python/pi.py", line 32, in <module> .getOrCreate() ^^^^^^^^^^^^^ File "/opt/bitnami/spark/python/lib/pyspark.zip/pyspark/sql/session.py", line 497, in getOrCreate File "/opt/bitnami/spark/python/lib/pyspark.zip/pyspark/context.py", line 51

I’m currently migrating from Apache NiFi 1.21 to NiFi 2.6.0. In my existing flow, I was using the InvokeGRPC processor, but I noticed that this processor has been removed in NiFi 2.x versions.I have a few questions:What is the exact reason for deprecating and removing InvokeGRPC in NiFi 2.x?Was it due to architectural changes, performance issues, or security concerns?If we take a fork of the NiFi 1.21 gRPC NAR and use it in NiFi 2.6.0, will that work?Are there any known performance risks with this approach?Could this introduce security vulnerabilities in the current or future releases?Is there an alternative recommended approach for gRPC integration in NiFi 2.x?For example, should we use ExecuteScript, custom processors, or any official replacement?Any insights or official guidance would be greatly appreciated. Thanks in advance!

I am setting up a 3-node Apache Kafka cluster using KRaft mode (3.9.0) with SCRAM-SHA-256 for inter-broker communication on the SASL_PLAINTEXT listener. The cluster is unable to form a quorum due to persistent SASL authentication failures.All configuration files and passwords have been synchronized. The exact error points to a handshake failure, but the settings appear correct.Environment and Node Details Kafka Version: 3.9.0 (KRaft mode)Security Protocol: SASL_PLAINTEXT (SCRAM-SHA-256)JAAS File Location: /opt/kafka/config/kafka_server_jaas.confNodes:Node 1: 172.20.24.155 (node.id=1)Node 2: 172.20.24.156 (node.id=2)Node 3: 172.20.24.157 (node.id=3)Errors The primary error is a SASL handshake failure, leading the client (connecting broker) to prematurely send an API request. This confirms that the broker is attempting to authenticate but is immediately rejected by the remote server.NODE 1 Logs (Example of persistent failures after configuration changes)Nov 13 13

Hello,During a security scan, two issues have been discovered in my Cloudera cluster (Cloudera Manager version 7.13.1, Cloudera Runtime 7.3.1).Zookeeper issue (Zookeeper version 3.8.1.7.3.1.0-197):On the nodes where zookeeper runs, I have the bellow issue:HTTP TRACE / TRACK Methods Allowed - Debugging functions are enabled on the remote web server - port 7000Mapreduce HSTS header:HSTS Missing From HTTPS Server / port 13562 Can you please assist on how to mitigate these issues? Thank you! 

Sqoop command is failing for Oracle database on CDP 7.1.9 cluster and same command works on different cluster of same versionThe only difference between working and non-working is storage, it might not be the main issue.sqoop command: sqoop import -libjars ./ojdbc8.jar -Doracle.jdbc.defaultNChar=true -Doracle.jdbc.OracleConnection.CONNECTION_PROPERTY_SET_ROW_PREFETCH_SIZE=10 -Doracle.jdbc.timezoneAsRegion=false -Doracle.sessionTimeZone=Asia/Kolkata -Doracle.jdbc.NLS_LANG=AMERICAN_AMERICA.AL32UTF8 -Doracle.jdbc.timezoneAsRegion=false --connect jdbc:oracle:thin:@hostname:1522:servicename --password $password --map-column-java CHAR_COL=String,DATE_COL=String,FLOAT_COL=Double,TIMESTAMP_COL=String,VARCHAR_COL=String,VARCHAR2_COL=String,NUMPERPS_COL=String,LONG_COL=String --username $username --as-avrodatafile --split-by FLOAT_COL --mapreduce-job-name SQOOP_SOURCE --columns CHAR_COL,DATE_COL,FLOAT_COL,TIMESTAMP_COL,VARCHAR_COL,VARCHAR2_COL,NUMPERPS_COL,LONG_COL --delete-target-dir --tab

I am facing below error all jobs are failing because of this.this is the error give guidance why it occurs because of this error all jobs failed also please share what other relevant information needed.Data integration service failed to create DIM instance because of the following error:jave.lang runtime exception java.IOException[error while authentication with endpoint. https://sitlxdvdlap097/kms/v1?op=GETDELEGATIONTOKEN&doAs=k164prda&renewer=K164PRDA%40SAIBSIT.COM]AUTHENTICATION FAILED, URL https://sitlxdvdlap097.saibsit.com:9494/kms/v1/?op==GETDELEGATIONTOKEN&doAs=k164prda&renewer==K164PRDA%40SAIBSIT.COM&user.name=K164PRDA, status=403 message: null ]

I am currently pursuing the CLOUDERA AI Technical Professional Accreditation (CAITP) certification. I have completed the first two courses successfully; however, I am unable to proceed to the third course because there is no “Launch” button displayed.Additionally, even though the first two courses show as completed individually, the overall certification progress is not updated and does not reflect the completed percentage.I would like to continue the remaining courses, but there is no option or path to proceed further.For reference, I have attached screenshots showing:Completed status of the first two coursesMissing launch button for the third courseLack of progress update on the overall CAITP certificationFor comparison, I have already completed the CLOUDERA AI Sales Professional Accreditation (CAISP) certification successfully without encountering this issue.Can you please assist me in resolving this and enabling access to the remaining course