Solved

Hello I have a flow where a queue is having say 250 entries but I am able to see only 100. I don't see any option to see the next sets of entries. How to view them as I can see the count to be increasing in queue but it displays oldest 100.

Hello I have configured certificate based authorization by using manager-authorizer in nifi.properties and then adding property name ="Initial Admin Identity" in <userGroupProvider> and <accessPolicyProvider> sections of authorizers.xmlNow once I do that and import admin cert (which I specified in authorizers.xml) then I can access NiFi and I can bring comonents/processors from Component Toolbar. I can also add users from the Global Menu "Users" section or Add Policies.Issue I am experiencing is that the user I added can only view the processors groups and can't modify them. Apart from this the whole component toolbar is grayed out so not accessible. I went to my admin account and added that user in some Policies but this did not help.I saw in some article that we need to right click on certain processor group and then go to "Manage Access Policies" but surprisingly for my admin account the Add User icon is gray and not able to click it. I have added two photos one sho

Hello I want to know one thing. Let's say we have a flow having a Handlehttprequest then a ExecuteSql processor and then a HandleHttpResponse for example. Daily we receive say 1000 calls to this flow. So where does flowfiles get stored? I see in provenance many many flowfiles. So do they purge or all get accumulated. I have less space so wanted to know.

Hello in NiFi can we have some restriction over users about what can they view and not edit other users processor groups. I want to have like say we have 5 persons then I want to give 5 user id and don't want one to interfere with others work like processor groups and possibly not even view other's processor groups.I was viewing one article (https://docs.cloudera.com/cfm/2.0.4/nifi-admin-guide/topics/nifi-creating-users-groups.html) mentioning "From the UI, select "Users" from the Global Menu." But I am not getting where is this Users option and also Global Menu which icon can be that.

Hello experts,We are running nifi 1.25.0 version in Kubernetes, single pod in cluster mode.Memory limit for the pod is 40G and JVM configured for Nifi is 28G.Now we are facing OOM error once in a while.When we checked nifi GUI System diagnostic it shows only 3G usage (Heap + non-heap)But when we checked at pod level memory usage for nifi process it shows very high usage.What else could be consuming so much memroy?This pod has only nifi container runningThanks in advance,Mahendra

Greetings to the community,I've recently joined as we're currently implementing EFM (Edge Flow Manager) and encountering issues with the installation of the MiNiFi agent on several Windows Server machines. These installation problems are preventing us from integrating the servers into the platform.Moreover, we're unable to uninstall the application either — in both installation and uninstallation attempts, we receive the same error message, which is blocking further progress.Any insights or recommendations from those who have faced similar challenges would be greatly appreciated.please check image attached to check the message.

Hi everyone,I need to know that can i add random user(not LDAPuser) in HUE without having AD user.Please see the atatched image.it's avaiable only LDAP useras well as i need to Assign Roles/Permissions in apache ranger too Cloudera manager version - CDP 7.7.1 Thank you!

Dear Experts,I would like to implement logging at the Processor Group level. Suppose we have two processor groups, A and B, and I want to maintain separate log files for each. When Processor Group A is started, its activities should be logged in a dedicated log file for Group A. Similarly, Processor Group B should have its own separate log file. How can I achieve this?Thank you in advance for your assistance. 

Please, I have a couple of questions. To put it in context, I need to capture only the flows that are running with errors, and I'm wondering how or what the recommended configuration would be for configuring the Site-to-Site Status Reporting Task controller for a 3-node cluster. As it is, it generates a lot of error messages and fills up my disk. Therefore, I want to suppress those messages with the correct configuration.node-1:8080SiteToSiteStatusReportingTask[id=743817f2-0199-1000-0000-000026cab202] EndpointConnectionPool[Cluster URL=[http://172.16.7.xxx:8080/nifi-api]] failed to communicate with Peer[url=nifi://node-3:2888,CLOSED] due to java.io.EOFExceptionnode-2:8080SiteToSiteStatusReportingTask[id=743817f2-0199-1000-0000-000026cab202] EndpointConnectionPool[Cluster URL=[http://172.16.7.xxx:8080/nifi-api]] failed to communicate with Peer[url=nifi://node-3:2888,CLOSED] due to java.io.EOFExceptionAll NodesSiteToSiteStatusReportingTask[id=743817f2-0199-1000-0000-000026cab202] Error r

Hello, Greetings to the entire community, I have a problem, we currently have a nifi cluster with version 1.11.4, I need to send alerts to datadog, and the SiteToSiteStatusReportingTask processor, it helps me obtain error information in the flows that we have developed as I understand, however this processor does not appear for the version of Nifi that we have, is there any other way to obtain the information of the flows with errors or any maintenance of updating or putting a patch nifi-site-to-site-reporting-nar-1.11.4.nar I would appreciate any guidance or support.

Experts,Provide me jolt transform for my json input shown belowInput json would be like below. This is just one record. Actually there can be multiple records{"Product": "4039","Plant": "1010","MRPArea": "","PlndIndepRqmtType": "VSE","PlndIndepRqmtVersion": "00","RequirementPlan": "","RequirementSegment": "","PlndIndepRqmtIsActive": "X","PlndIndepRqmtPeriod": "20250502","NoWithdrawal": "","DeleteOld": "","PeriodType": "2","PlannedQuantity": "20.000","UnitOfMeasure": "PC","ProductionVersion": ""}Output what i am expecting as below{"Product": "4039","Plant": "1010","MRPArea": "","PlndIndepRqmtType": "VSE","PlndIndepRqmtVersion": "00","RequirementPlan": "","RequirementSegment": "","PlndIndepRqmtIsActive": "X","PlndIndepRqmtPeriod": "20250502","NoWithdrawal": "","DeleteOld": "","to_PlndIndepRqmtItem": [{"Product": "4039","Plant": "1010","MRPArea": "","PlndIndepRqmtType": "VSE","PlndIndepRqmtVersion": "00","RequirementPlan": "","RequirementSegment": "","PlndIndepRqmtPeriod": "20250502"

Hey, communiy!We have a NiFi Cluster 1.18.0 (still so, yeap, sorry) and next issue do my mind.Simple flow, where we read data from Kafka with ConsumeKafka Processor and process it with EvaluateJsonPath after.One time we can see that queue between processors are stuck with data and nothing happens while EvaluateJsonPath or whole canvas not restared (right click -> stop -> start):Connection settings:Back Pressure Threshold: 10 000;Size Threshold: 1 GbLB Strategy: Round RobinQueue threshold exceeded only on 3rd cluster node: Additional configuration data:Maximum timer driven thread count: 400112 cores per node (2 CPU, 28 physical cores, 112 total with hyperthreading) So, as soon as I stop and start canvas -- all works fine again. Why it happens? How can I find the reason? 

Hello,We have got a new requirement. Earlier the users were sending Soap request and our existing app was processing the soap request and then returning a soap response. We now want to do this using NiFi. We want users to be still sending the soap request and we will receive it in NiFi and then process it and return back a Soap response. Is there any way I can do this in NiFi?

I am running into a problem with my DBCPConnectionPool service in Nifi and am hoping that someone has some experience with this. I'm using Nifi 2.0.0-M2.My current Database Connection URL isjdbc:postgresql://myurl:5432/mydatabase?currentSchema=public&stringtype=unspecifiedThis works great with no issues. However, I want to add another schema in there so I don't have to specify the new schema in my queries throughout all my nifi flows. For example, I was hoping this would work:jdbc:postgresql://myurl:5432/mydatabase?currentSchema=myschema,public&stringtype=unspecifiedBut when I do a query like "select * from mytable" with ExecuteSQL, I get an error: org.postgresql.util.PSQLException: ERROR: relation "mytable" does not exist If I change the query to "select * from myschema.mytable", it works.Chatgpt recommended I try this, but it also didn't work:jdbc:postgresql://myurl:5432/mydatabase?options=-c%20search_path=myschema,public&stringtype=unspeci

Hi,The repository https://repository.cloudera.com/artifactory/cdh-releases-rcs is not working and obtain the "Error 500 Internal Server Error"When I try to download the library https://repository.cloudera.com/artifactory/cdh-releases-rcs/org/apache/zookeeper/zookeeper/3.4.5-cdh5.4.0/zookeeper-3.4.5-cdh5.4.0.jar, I get a 500 error.On Friday the 12th, it was working.

Hi,We have a flow created with ListSFTP , we have it scheduled it once per day, but for some reason if files are not available at source, we would like to Stop it and execute Run Once and Start the Primary Processor again.These step work fine from NiFi UI but when we try to use NiFi Api (/processors/{id}/run-status) to stop the Primary processor, the Processor wont stop, we are passing the latest Version Id and Client Id as well.any suggestion on how to fix this ?

Environment:- CDP Public Cloud (AWS)- DataHub cluster with CDH 7.3.1- Custom service: 'Service_Name'- Deployment via cluster template + pre/post recipesProblem:Cluster creation fails with the error:```No service type 'Service_Name' available for cluster with version 'CDH 7.3.1'```What we have done:- Created a valid 'Service_Name.jar' with:- 'descriptor/service.sdl'- 'cloudera-service.json' (includes '"cdhVersion": { "min": "7.3.1"}')- Used **pre-recipe** to download and place CSD and parcel files in '/opt/cloudera/csd/' and '/opt/cloudera/parcel-repo/'- Used **post-recipe** to distribute and activate parcels via Cloudera Manager API- Verified JAR contents and permissions- Still getting the same error during cluster creationERROR:Installation of CDP with Cloudera Manager has failed. Please find more details on Cloudera Manager UI. Failed command(s): ClusterTemplateImport(id=1546337329): java.lang.IllegalArgumentException: No service type 'Service_Name' available for cluster with version

How to run spark df.write inside UDF called in rdd.foreach or rdd.foreachpartitionI.e. spark session object inside executor.

Hello,Based on this example in Iceberg doc I tried to create a table using a custom hadoop catalog.Using Hue in CDP 7.2 I performed SET iceberg.catalog.hadoop_cat.type=hadoop;SET iceberg.catalog.hadoop_cat.warehouse=s3a://my-bucket/hadoop_cat_warehouse;CREATE TABLE iceberg_hadoop_cat_table (id Integer,name String) STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler' TBLPROPERTIES ('iceberg.catalog'='hadoop_cat'); But received Error while compiling statement: FAILED: Execution Error, return code 40000 from org.apache.hadoop.hive.ql.ddl.DDLTask. java.lang.IllegalArgumentException: Table location not set Did I miss something ? Is CDP suporting that ?Thanks,Laurent

<userGroupProvider> <identifier>file-user-group-provider</identifier> <class>org.apache.nifi.authorization.FileUserGroupProvider</class> <property name="Users File">./conf/users.xml</property> <property name="Initial User Identity 1">nifi</property> <property name="Initial Group Identity 1"></property></userGroupProvider><accessPolicyProvider> <identifier>file-access-policy-provider</identifier> <class>org.apache.nifi.authorization.FileAccessPolicyProvider</class> <property name="User Group Provider">file-user-group-provider</property> <property name="Authorizations File">./conf/authorizations.xml</property> <property name="Initial Admin Identity">nifi</property> <property name="Initial Admin Group"></property> <property name="Node Identity 1"></p

Hello All.What problems can there be when copying data between two clusters with different major versions if you use hdfs://... instead of webhdfs://...hadoop distcp hdfs://<namenode>:<port> hdfs://<namenode>Examle from documetntation -Copying between major versionsRun the distcp command on the cluster that runs the higher version of Cloudera, which should be the destination cluster. Use the following syntax:hadoop distcp webhdfs://<namenode>:<port> hdfs://<namenode>Note the webhdfs prefix for the remote cluster, which should be your source cluster. You must use webhdfs when the clusters run different major versions. When clusters run the same version, you can use the hdfs protocol for better performance.For example, the following command copies data from a Cloudera source cluster named example-source to another Cloudera version destination cluster named example-dest:hadoop distcp webhdfs://example-source.cloudera.com:8020 hdfs://example-dest.clouder

Dear Support Team,I hope this message finds you well.I am currently working on setting up Trino to query data from Hive. However, I am encountering difficulties related to Kerberos authentication.  Despite following available documentation, I have not been able to successfully connect to the Hive. I think there is something wrong with my KERBEROS settings. When I use telnet or ping, I can connect to HIVE correctly, but when I use trino CLI,  it shows that it is socket is closed by peer. I would appreciate your guidance on:1. I would like to ask whether I need to set up Kerberos separately for trino connection? 2. How can I get relevant information about kerberos authentication, such as keytab.Here are some details about my setup:（I think maybe at least this information is needed, but I don't know where to get them as following）1.hive.metastore.client.principal=trino.example.com 2.hive.metastore.client.keytab=trino.keytab .Please let me know if there are any files I shoul

Hello All,I am new to Apache Nifi where we have a requirement to convert a user input date into Nifi date format and use it in an sql query running inside an executeSQL processor.I am facing challenge while converting the user input date, Any help would be really appreciated.Below are my flowfilesGenerateFlowfile with a Custom Date properties with value as "2023-01-15 10:30:00"UpdateAttribute using the Custom Date as End_date with value as ${'Custom Date':plus(864000000):format('yyyyMMddHHmmss')}ExecuteSQL using End_date in Oracle query as " TO_DATE ('${End_Date}', 'YYYYMMDDHH24MISS')"Also is there any better way to handle this. 

Hi, I am changing my organization. Currently i have my organization email id in my account which is also my user name. I have to change it to a new email address. How can i change it? Can you please help me change it?

I have deployed a single node cluster in my lab.  It consists of two virtual machines implemented in VMware: one manager and one host.Trying to Add Cloudera Management Service to address warning “Unable to issue query: could not connect to host monitor”.This is a test of my database:    [root@cloudera-manager ~]# hostname -A    cloudera-manager.interop.local cloudera-manager    [root@cloudera-manager ~]# psql -d scmdb -U scm -W    Password for user scm:    psql (10.23)    Type "help" for help.I entered the following in the Setup Database  GUI:    Type: PostgreSQL    Database Hostname: cloudera-manager.interop.local    Database Name: scmdb    Usename: scm    Password: xxxxxxxSelecting Test Connection fails.Any ideas?Dalier.