Ask the Community

I have created a mysql database (scm) & user (scm@localhost) on a RHEL7 server. CM server version is cloudera-manager-server-7.6.7-36957245.el7.x86_64; mysql version is mysql Ver 8.0.42 for Linux on x86_64 (MySQL Community Server - GPL). DB user has following grants...mysql> grant select, insert, update, delete, create, drop on scm.* to 'scm'@'localhost';When the scm_prepare_database.sh script is executed, it returns "All done, your SCM database is configured correctly!"  HOWEVER, multiple database tables are NOT created & the cloudera-scm-server.service will NOT stay started.cloudera-scm-server.log returns multiple errors likeCaused by: java.sql.SQLSyntaxErrorException: Table 'scm.CM_VERSION' doesn't existCaused by: org.hibernate.exception.SQLGrammarException: could not extract ResultSetCaused by: javax.persistence.PersistenceException: org.hibernate.exception.SQLGrammarException: could not extract ResultSetCaused by: java.lang.RuntimeException: Unable t

i have a JSON payload which contains latest and old orders. The data should be compared and output New, update, deleted. How can do it do that in NIFI?.  "latest_orders": [    { "order_id": "1001", "customer": "Alice", "amount": 250 },    { "order_id": "1002", "customer": "Bob", "amount": 300 },    { "order_id": "1004", "customer": "Diana", "amount": 150 }  ],  "old_orders": [    { "order_id": "1001", "customer": "Alice", "amount": 200 },    { "order_id": "1002", "customer": "Bob", "amount": 300 },    { "order_id": "1003", "customer": "Charlie", "amount": 400 }  ]}Output:    { "order_id": "1001", "customer": "Alice", "amount": 250, "Action":"UPDATE" },    { "order_id": "1002", "customer": "Bob", "amount": 300 ,"Action":"UPDATE"},    { "order_id": "1004", "customer": "Diana", "amount": 150,"Action":"NEW" },    { "o

Hi team,I am currently facing issue connecting to Hive database using python libraries (pyodbc, pyhive, impyla), the Cloudera driver and Apache Knox. While the network level connection to the relevant host and port is successful. (verified via ncat, telnet), I am unable to complete the connection and authenticate through these tools. Connection details:Driver=/opt/cloudera/hiveodbc/lib/64/libclouderahiveodbc64.so;""Host=nonprodm01.cdslindia.com;""Port=8443;""Schema=default;""TransportMode=http;""HTTPPath=/gateway/default/hive""SSL=0;"  # Enable SSL if Knox uses HTTPS"AuthMech=3;"  # For username/password authentication; use 1 for no auth or 2 for Kerberos"UID=username;""PWD=password;"Knox gateway is used to access HiveServer2Error message: Connection failed: "('HY000', '[HY000] [Cloudera][DriverSupport] (1110) Unexpected response received from server. Please ensure the server host and port specified for the connection are correct and confirm if SSL should be enabled for the c

Hello!Following the advice from this post: https://community.cloudera.com/t5/Support-Questions/Accessing-NIFI-Metrics-endpoint-for-Prometheus-without/m-p/399710#M250535I'm trying to set up certificate authentication/authorization from prometheus server to nifi, but getting 403 Forbidden. In Nifi user log i can see that authentication CN of certificate is successful, further  comes the 403 error:Prometheus scrape config: Can you please tell what is wrong here?

We've encountered an issue within Cloudera Object Store (powered by Apache Ozone) (1.4.0 and 1.4.1) which we believe are related to HDDS-9967A fix for this particular issue has now been backported to the 'apache:ozone-1.4' project as part of pull request https://github.com/apache/ozone/pull/9066How do we update the Cloudera Runtime or Ozone parcel version (1.4.0.7.1.9.1054-4) to include this fix? Upgrading to the latest 1.4.2 would be ideal.Thanks Stephen

Hi,We have a NiFi cluster which spans two physical locations, Zone A and Zone B.  We are clustering across locations for ease of management.  In each location we have hosts running NiFi acting as data collectors, enriching the data and then sending to the main cluster via site-2-site (using remote processing groups).My question is:We want data originating in Zone 1 to only hit the NiFi Cluster nodes in Zone 1.  Using site-2-site, even if we only configure the Zone 1 nodes in the RPG, NiFi will report the presence of the Zone 2 nodes hence data originating from Zone 1 will cross the WAN and end up in Zone 2.  Does NiFi have any concept of zones?We currently achieve this by configuring a firewall rule to block outgoing traffic to Zone 1 nodes from Zone 2 (and vice-versa), but this produces warnings in the RPG.Any thoughts or ideas would be appreciated.  Thanks 

Hi Team, We have the following issue, we are using nifi work flow, in which we are trying to execute impala, before implementing nifi  execute script, we tested the script by using jython to test the feasibility of the connections we established is successful and able to get the count in impala, however when using the same connection configuration in nifi execute script processor, we get the java class path not found, the impala jdbc connection is placed in a customized location, impalajdbc41.jar file is not a nar, file, and the customized location is readable, as confirmed by the support and if placed in the nar file location will Please find the logs for reference as an attachment.2025-09-29 15:31:08,495 ERROR org.apache.nifi.processors.script.ExecuteScript: ExecuteScript[id=49498cfb-70fc-1ebf-ffff-fffff0fa2dc8] Processing failedorg.apache.nifi.processor.exception.ProcessException: javax.script.ScriptException: java.lang.ClassNotFoundException: java.lang.ClassNotFoundException:

Hello everyone,Is it possible to limit the memory for executing queries for an LDAP group in CDP 7.1.7? This would allow creating a memory pool for each group so that if a user in that group launches a memory-intensive query, it does not impact other users in the cluster.BR!

Hitting a weird issue in Nifi that I can't get my head around.I am consuming AVRO records from Kafka with a timestamp field. The definition is { "name" : "timestamp", "type" : { "type" : "long", "logicalType" : "timestamp-millis" } },If I use a JSON record writer then in the queue viewer I can see the field contains the original long value. However if I use an AVRO writer the field is formatted as a date/time value, e.g. 2025-09-26T15:29:44.466Z.I assumed this was just pretty-printing in the queue viewer. But if I then send those flowfiles to a ConvertRecord process with an AVRO reader and a FreeText writer the timestamp field is now displayed as YYYY-mm-DD HH:MM:SS. Which means I've lost precision and I need to use expression language to re-parse the field to convert it back to a long value.So, I'm forced to us JSON as the intermediary format. If I use AVRO I loose precision.This is Nifi 2.5.

Hi,I read from the release note that the encrypt-config.sh tool from nifi-toolkit has been removed in NiFi 2.0 due to various considerations(https://issues.apache.org/jira/browse/NIFI-13414). I am wondering is there any alternatives to this tool? We heavily relied on this encrypt-config.sh tool to encrypt plain-text passwords in nifi.properties, like nifi.sensitive.props.key, nifi.security.keyPasswd. Now with this tool gone, do we just have to accept the fact that plain-text passwords will remain plain-text?Thanks in advance!!

Hello I have a flow where a queue is having say 250 entries but I am able to see only 100. I don't see any option to see the next sets of entries. How to view them as I can see the count to be increasing in queue but it displays oldest 100.

Hello I have configured certificate based authorization by using manager-authorizer in nifi.properties and then adding property name ="Initial Admin Identity" in <userGroupProvider> and <accessPolicyProvider> sections of authorizers.xmlNow once I do that and import admin cert (which I specified in authorizers.xml) then I can access NiFi and I can bring comonents/processors from Component Toolbar. I can also add users from the Global Menu "Users" section or Add Policies.Issue I am experiencing is that the user I added can only view the processors groups and can't modify them. Apart from this the whole component toolbar is grayed out so not accessible. I went to my admin account and added that user in some Policies but this did not help.I saw in some article that we need to right click on certain processor group and then go to "Manage Access Policies" but surprisingly for my admin account the Add User icon is gray and not able to click it. I have added two photos one sho

Hello I want to know one thing. Let's say we have a flow having a Handlehttprequest then a ExecuteSql processor and then a HandleHttpResponse for example. Daily we receive say 1000 calls to this flow. So where does flowfiles get stored? I see in provenance many many flowfiles. So do they purge or all get accumulated. I have less space so wanted to know.

Hello in NiFi can we have some restriction over users about what can they view and not edit other users processor groups. I want to have like say we have 5 persons then I want to give 5 user id and don't want one to interfere with others work like processor groups and possibly not even view other's processor groups.I was viewing one article (https://docs.cloudera.com/cfm/2.0.4/nifi-admin-guide/topics/nifi-creating-users-groups.html) mentioning "From the UI, select "Users" from the Global Menu." But I am not getting where is this Users option and also Global Menu which icon can be that.

Hello experts,We are running nifi 1.25.0 version in Kubernetes, single pod in cluster mode.Memory limit for the pod is 40G and JVM configured for Nifi is 28G.Now we are facing OOM error once in a while.When we checked nifi GUI System diagnostic it shows only 3G usage (Heap + non-heap)But when we checked at pod level memory usage for nifi process it shows very high usage.What else could be consuming so much memroy?This pod has only nifi container runningThanks in advance,Mahendra

Hi All,I hope you are doing well.I would like from you the support in explain these limitations.Limitations with Analytical ApplicationsThis topic lists all the limitations associated with the Applications feature.Port availabilityCloudera AI exposes only 2 ports per workload. Therefore, you can run a maximum of 2 web applications simultaneously, on these ports:CDSW_APP_PORTCDSW_READONLY_PORTBy default, third-party browser-based editors run on CDSW_APP_PORT. Therefore, for projects that are already using browser-based editors, you are left with only one other port to run applications on: CDSW_READONLY_PORT. How can I integrate these ports to my web application?Best wishes,Salim 

Hello all,I'm currently running Nifi 1.23.2 in a 3 nodes cluster.  I would received the following message when I tried to login.  I would get kick out then this message will appear.  I verified I have sticky session enabled on SSLSession. Its also configured in round robin DNS.  Is there anything else I can look at thats contributing to this issue?   Thanks!

Hi,we using the cdh_version: 7.1.9 and cm_version: 7.11.3We want to deploy the cloudera manager with the json template file for our test automation. We want to use the rest API call "/cm/importClusterTemplate?addRepositories=true".But we get the following error when we deploy the template json file.{"message": "The following host(s) are not available: hdfs-11.vmlocal,hdfs-14.vmlocal,hdfs-12.vmlocal,hdfs-13.vmlocal."The hdfs nodes are available and can be ping or ssh from the cloudera host.The following packages are installed on each hdfs node:cloudera-manager-daemons-7.11.3.7-52024171.el8.x86_64cloudera-manager-agent-7.11.3.7-52024171.el8.x86_64openjdk8-1.8.0_372_cloudera-1.x86_64The  cloudera-scm-agent running and the server_host entry in /etc/cloudera-scm-agent/config.ini points to the cloudera manager host (server_host=10.0.100.101)We have read the cloudera documentation multiple times.Does anyone has an idea why this error appears or how we can verify that the cloudera manager

Greetings to the community,I've recently joined as we're currently implementing EFM (Edge Flow Manager) and encountering issues with the installation of the MiNiFi agent on several Windows Server machines. These installation problems are preventing us from integrating the servers into the platform.Moreover, we're unable to uninstall the application either — in both installation and uninstallation attempts, we receive the same error message, which is blocking further progress.Any insights or recommendations from those who have faced similar challenges would be greatly appreciated.please check image attached to check the message.

Hello,I have noticed in one of the charts (HDFS IO) available in CM dashboard for HDFS that runs below query have got values pulled from 2 separate entities for the same metric. Attached are the 3 screenshots for reference. 2 graphs showing values from individual entities and 1 the combined graph available in CM that triggers below query.select total_bytes_read_rate_across_datanodes, total_bytes_written_rate_across_datanodes where category = SERVICE and serviceType = HDFS and clusterId = "1"Two different entities are hdfs and hdfs:<cluster_name>I have also verified the response from below API call, and it does returns both entities and both have latest data.http://<cm-host>:7180/api/v19/timeseries?query=SELECT%20total_bytes_written_rate_across_datanodes%20WHERE%20category=SERVICEWhat do they mean?Where are they coming from?How to get it fixed so a single value is returned for the metric?Kindly advise.ThanksSnm1523

Dear Cloudera support team ,I’ve noticed that the Query tab in Impala is not displaying queries run from Hue immediately; they only appear after a delay. Could you please let me know what the issue might be and how it can be resolved? BR,Modhi

Hi everyone,I need to know that can i add random user(not LDAPuser) in HUE without having AD user.Please see the atatched image.it's avaiable only LDAP useras well as i need to Assign Roles/Permissions in apache ranger too Cloudera manager version - CDP 7.7.1 Thank you!

Dear Experts,I would like to implement logging at the Processor Group level. Suppose we have two processor groups, A and B, and I want to maintain separate log files for each. When Processor Group A is started, its activities should be logged in a dedicated log file for Group A. Similarly, Processor Group B should have its own separate log file. How can I achieve this?Thank you in advance for your assistance. 

Please, I have a couple of questions. To put it in context, I need to capture only the flows that are running with errors, and I'm wondering how or what the recommended configuration would be for configuring the Site-to-Site Status Reporting Task controller for a 3-node cluster. As it is, it generates a lot of error messages and fills up my disk. Therefore, I want to suppress those messages with the correct configuration.node-1:8080SiteToSiteStatusReportingTask[id=743817f2-0199-1000-0000-000026cab202] EndpointConnectionPool[Cluster URL=[http://172.16.7.xxx:8080/nifi-api]] failed to communicate with Peer[url=nifi://node-3:2888,CLOSED] due to java.io.EOFExceptionnode-2:8080SiteToSiteStatusReportingTask[id=743817f2-0199-1000-0000-000026cab202] EndpointConnectionPool[Cluster URL=[http://172.16.7.xxx:8080/nifi-api]] failed to communicate with Peer[url=nifi://node-3:2888,CLOSED] due to java.io.EOFExceptionAll NodesSiteToSiteStatusReportingTask[id=743817f2-0199-1000-0000-000026cab202] Error r

Hi Team, Since couple of days we are finding issues in CDP Public Cloud clusters of both Azure and AWS , where Hive service is not working. Issue happening for both existing and newly deployed clusters. Please find the error message below. ERROR: ERROR HiveMetaStore[main]: MetaException(message:java.io.IOException: org.apache.knox.gateway.shell.ErrorResponse: https://xxxx.xxxx.cloudera.site:8444/gateway/dt/knoxtoken/api/v1/token: HTTP/1.1 500 Server Error)Caused by: org.apache.hadoop.service.ServiceStateException: java.io.IOException: org.apache.knox.gateway.shell.ErrorResponse: https://xxxxx.xxxx.cloudera.site:8444/gateway/dt/knoxtoken/api/v1/token: HTTP/1.1 500 Server Error