Ask the Community

We would like to automate the installation  / configuration of the Cloudera Manager instead of using the GUI.We are using ansible for the automation steps.We have uploaded the license file with the REST API call in ansible. This works fine. Now we try to import a cluster template that we took from another setup. We did some modification at that template and start to do the import via ansible. The import failed with an error that the cluster host can't are not available. The cluster hosts are available, they can be ping from the cloudera manager host.The cloudera-scm-agent are installed at cluster host and up and running.  The /var/log/cloudera-scm-server log shows two java exceptions and the following message The following host(s) are not available: test-22.vmlocal,test-23.vmlocal,test-21.vmlocal,test-24.vmlocal.2025-07-10 15:50:44,353 INFO scm-web-75:com.cloudera.api.ApiExceptionMapper: Exception caught in API invocation. Msg:The following host(s) are not

I am using MariaDB and CMS is up and running, but I am getting the following message in the logs for Reports Manager. I actually deleted the rman user, recreated the database, I logged into mysql with that user, but I am still getting access denied to the database. Any ideas?com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@619cdccb -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (5). Last acquisition attempt exception: java.sql.SQLException: Access denied for user 'rman'@'dmidlkprdls01.svr.luc.edu' (using password: YES) at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:121) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:113) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:861) at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionI

Hello @MattWho ,I'm new to NIFI and Nifikop but I'm using terraform and helm to deploy NIFI on AKS with nifikop, it works well when configured with single-user-provider and overiding nifiproprites with oidc for azure AD, but when I tried to deploy it with file-user-groups provider all the pods kept running and then getting this error : Caused by: java.io.FileNotFoundException: ../data/users.xml (No such file or directory) at java.base/java.io.FileOutputStream.open0(Native Method) at java.base/java.io.FileOutputStream.open(FileOutputStream.java:289) at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:230) at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:179) at jakarta.xml.bind.helpers.Absand this is my helm chart :  # nifi.security.user.login.identity.provider=single-user-provider # nifi.security.user.authorizer=single-user-authorizer nifiProperties: overrideConfigs: | nifi.in

Hi everyone,I have been facing intermittent issues with a delete statement on a iceberg table v2.Below is the error I am having and I dont know what is causing it.Had anyone gone through the same issue?Thank you [HiveServer2-Background-Pool: Thread-1255]: Vertex failed, vertexName=Reducer 2, vertexId=vertex_1752069399412_0193_1_01, diagnostics=[Task failed, taskId=task_1752069399412_0193_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Node: xxxxx-compute0.xxxxx.ysv060.a0.cloudera.site/10.XX.XX.XX : Error while running task ( failure ) : attempt_1752069399412_0193_1_01_000000_0:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing vector batch (tag=0) (vectorizedVertexNum 1)at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:351)at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:280)at org.apache.tez.runtime.LogicalIOP

Does anyone know where to find the Maven repository for HDP 3.1.5.0-152?I want to compile HDP 3.1.5.0-152, but I can't find the corresponding Maven repository.At https://repo.hortonworks.com/content/repositories/releases/, only dependencies for other HDP versions are available.Thanks in advance.

Hi all,I've created a custom service in Cloudera Manager (ABC_ROLE) and defined a startRunner that calls a control.sh script. This script eventually runs a start_and_monitor function that starts two internal services and enters a monitoring loop.service.sdl Snippet:"roles": [ { "name": "ABC_ROLE", "startRunner": { "program": "scripts/control.sh", "args": ["start"], "environmentVariables": { "LOG_LEVEL": "${log_level}" } }, "stopRunner": { "relevantRoleTypes": ["ABC_ROLE"], "runner": { "program": "scripts/control.sh", "args": ["stop"] } } }],"rollingRestart": { "workerSteps": { "roleName": "ABC_ROLE", "bringDownCommands": ["Stop"], "bringUpCommands": ["Start"] }}control.sh Snippet:control.shstart_service() { echo "start_service() called" sleep 60 #Adding sleep to test exec ${SERVICE_PARCEL_HOME}/service/scripts/service_daemon.sh start_and_monitor}if [ "$CMD" == "start" ]; then start_serv

Hi All,I am trying to secure NIFI with Keycloak based access control. I am able to set up the admin account and login successfully. Next step, I want to create a group access policy that will have most of the admin privileges, however not via UI.  Is it possible to provide custom-authorizations.xml  and custom-users.xml on NIFI Helm where i specify my group and policy for it and still keep the initial admin user being generated as well? So basically, the content of these custom files should be appended to the generated users.xml and authorizations.xml file. I tried couple of tricks and hacks such as mounting my custom file under the name users.xml and authorizations.xml , this will basically replace auto-generated users.xml and authorizations.xml , hence it does not include admin user but only the group/users I mentioned. I tried to manually add my admin user to the file as well but in this case, root process-group uuid needs to be specified in the po

We have installed Cloudera Manager 7.13.1 and CDP Private Cloud Base 7.3.1 on Red Hat Enterprise Linux 9.4.However, we are unable to find a supported Spark 3 parcel that is compatible with this combination. We have reviewed the Cloudera Support Matrix but could not find any officially supported Spark 3 version for CDP Private Cloud Base 7.3.1.We attempted to use the following Spark3 parcels:SPARK3-3.3.2.3.3.7190.0-91-1.p0.45265883SPARK3-3.3.0.3.3.7180.0-274-1.p0.31212967Displays the following error when unpacking spark3 parcelsThere was an error communicating with the server:CDH 7.3.1 replaces SPARK3 3.3.2.3.3.7190.0-91-1.p0.45265883.

We’ve set up Cloudera Manager 7.13.1 along with CDP Private Cloud Base 7.3.1 on RHEL 9.4. As part of our environment validation, we’re looking to include Apache Spark 3 in the stack. However, we haven’t been able to locate a supported Spark 3 parcel that’s officially compatible with this combination.We’ve reviewed the Cloudera Support Matrix, but couldn’t find a clear reference for Spark 3 support with these specific versions of CDP and CM.According to a recent blog post,  Announcing availability of Spark 3.5 on Cloudera Unified Runtime which also directs the reader to What's New in Apache Spark (7.3.1). Apparently Spark 3 is supported by 7.3.1 and Spark 3.5 by 7.3.1 SP1.Does Spark still get installed separately or is it included with the base runtime?Should the support matrix be updated to reflect support for Spark 3? 

i am installing cloudera cdh  6.3.3 with isilon but when adding yarn i am facing this error  any help i will be grateful

HI, I have recently configured Kafka services on our Big Data Cluster. However, I am encountering the following issues on the Kafka broker and while establishing Kafka connections:Issues Observed:Broker error:  Bad : There are 81 offline partitions.kafka.controller.ControllerEventManager$ControllerEventThread.$anonfun$doWork$1(ControllerEventManager.scala:131)  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)  at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:31)  at kafka.controller.ControllerEventManager$ControllerEventThread.doWork(ControllerEventManager.scala:131)  at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)kafka connect error: [Worker clientId=connect-1, groupId=connect-cluster] Uncaught exception in herder work thread, exiting: org.apache.kafka.common.errors.TimeoutException: Failed to get offsets by times in 30000msCloudera Bigdata Cluster configuration:Totol data node 7:master nodes: 3and edge node 2kafka configuration:Broker

I try to deploy and configure a CDP cluster over the CM RestAPI. However, I'm running into different problems around this. Starting with where to find values, e.g. /clusters/{clusterName}/parcels/products/{product}/versions/{version} where to find {product} and {version} for that.Or forcurl -X POST -u admin:admin -i -H "Content-Type: application/json" -d '{ "items" : [ { "ipAddress" : "{ipAddress1}", "hostname" : "{hostname1}" }, { "ipAddress" : "{ipAddress2}", "hostname" : "{hostname2}" } ] }' http://{cmIpAddress}:7180/api/v54/clusters/{clusterName}/hostsit returns following error although the Rest API documentation claims, that ipAddress is a required attributeHTTP/1.1 400 Bad Request[...]{"message" : "Unrecognized property: 'ipAddress'"}If I use the API Documentation on the CM it also does not fill out the information automatically, which would have been really helpful. So my question: could you provide the steps how to deploy and configure a CDP cluster with the CM Rest A

Hello,After upgrading CDP Private Cloud Base**from 7.1.7-1.cdh7.1.7.p2000.37147774 to 7.1.9-1.cdh7.1.9.p1013.58158856**,we can no longer access Hive databases through Hue behind Knox(Authentication backend: `KnoxSpnegoDjangoBackend### Symptoms1. Hue UI displaysSnippet type 1 is not configured.`).2. Hive databases and tables are no longer listed.3. Hue logs (`runcpserver.log`) showTypeError: Object of type WSGIRequest is not JSON serializable### Context* Hue is managed by Cloudera Manager; authentication is Knox → LDAP (Active Directory).* Hive is accessed through Knox (HTTPS 8443).* OS: RHEL 8.6; Python 3.8 shipped with the parcel.**Knox gateway log** records the following cookie-related error(ResponseProcessCookies.java:processCookies(130)) - Invalid cookie header:"Set-Cookie: csrftoken=1pPR9AOs9sRQaRGMxEws09bexNRMYg7W8SgJZGAlUzkBHZInYBs3IOaLBFisJjZj;expires=Tue, 12 May 2026 09:13:19 GMT; HttpOnly; Max-Age=31449600; Path=/;SameSite=Lax; Secure". Invalid 'expires' attribute: Tue, 12 Ma

Hi I have a 3 node Apache Nifi cluster setup, which is managed by a 3 node zookeeper cluster. The dev cluster worked fine, with one node frequently dropping off with us having to sometimes manually restarting the node, after renaming its flow.xml.gz and flow.json.gz, after which the node started up fine and connected to the cluster. But today, after 1 node went down, it wouldnt connect back to the cluster (even after renaming the flow gzs). Within some minutes another node disconnected from the cluster, and the last node which was the primary at that stage threw a socket time out, so I manually restarted it, and it wont startup throwing  Invalid State Cannot replicate request to Node oooo-nifiat01.yy.xxx.local:0000 because the node is not connected with the nifi-user.log complaining of  o.a.n.w.a.c.IllegalClusterStateExceptionMapper org.apache.nifi.cluster.manager.exception.IllegalClusterStateException: The Flow Controller is initializing the Data Flow.. Returning Conflict re

Hi, We are trying to setup auditing for HDFS using ranger audit with configuration as in Cloudera Manager HDFS -> Configuration Tab in NameNode Logging Advanced Configuration Snippet as :log4j.appender.RANGER_AUDIT=org.apache.log4j.DailyRollingFileAppender log4j.appender.RANGER_AUDIT.File=/var/log/hadoop-hdfs/ranger-hdfs-audit.log log4j.appender.RANGER_AUDIT.layout=org.apache.log4j.PatternLayout log4j.appender.RANGER_AUDIT.layout.ConversionPattern=%m%n log4j.logger.ranger.audit=INFO,RANGER_AUDIT,SYSAUDIT log4j.appender.SYSAUDIT=org.apache.log4j.net.SyslogAppender log4j.appender.SYSAUDIT.threshold=INFO log4j.appender.SYSAUDIT.syslogHost=remote-host:port log4j.appender.SYSAUDIT.layout=org.apache.log4j.EnhancedPatternLayout log4j.appender.SYSAUDIT.layout.conversionPattern={ "log4j_tz_offset": "%d{Z}" }PR3N0RM %d{MMM dd HH:mm:ss} ${hostName}HDFS:%m%n log4j.appender.SYSAUDIT.filter.a=org.apache.log4j.varia.LevelRangeFilter log4j.appender.SYSAUDIT.filter.a.LevelMin=INFO log4j.appende

Hello everyone.So I have this problem while running Hive, for example I want to create external table for my dataset, and instead of executing it, shows this error:"FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient"it's quite basic, but for the life of me, I can't seems to be able to fix it. I even deleted and redownloaded the Hive client and change versions, nothing change. Anyone care to lend a hand?

Hi Sir,I am using apache nifi-2.0.0-M1 version. I am using window 10. I have some work. I need to export these task & import to production. But create template is missing.Here is image of process group:Process group configuration:How to export process group & import to other apache nifi using same version?  

 Hello Folks,I want to combine 3 FlowFiles using a UNION statement. I’ve already transformed the 3 FlowFiles to have the same schema and would like to merge them into a single content, structured like this:FlowFile1 ContentUNION ALLFlowFile2 ContentUNION ALLFlowFile3 ContentI’d appreciate your help with this.

Hi , My requirement is to consume all records from kafka topic (e.g. 1 lakh) in one go by using ConsumeKafkaRecord processor which is scheduled to run once 4 hours as i need to merge all data and create a parquet file and store in HDFS path using putHDFS processor .if i consume all incoming data and then use a merge content processor and run it once in 4 hours then issue comes is all data stays in nifi queued for 4 hours taking memory , so for avoiding that i wanted to use ConsumeKafkaRecord with scheduler (4 hrs) but its not able to consume all record .How to solve this issue ?

Hi Everyone, Can you help me please.In my cluster have 80% used  storage each nodes, I want to clean up each node but I am very worried if deleting data from /hadoop/hdfs/data folder will affect HDFS cluster. see below capacity left 1TB but capacity used 2TB, but in not have table data in my clusterNot have table in my cluster like belowplease suggestions for case, Can I to remove file in folder /hadoop/hdfs/data? please tell me to step by step to remove it 

hello cloudera community Is it possible to know the date and time that a kudu table was accessed/read?i know there is "transient_lastDdlTime" field in kudu table but that field is updated only when the table is updated.

Hi, I have a fresh install of Cloudera Manager and it was running fine until I rebooted the server. The only thing I changed was the hosts file, but I don't think that has anything to do with it (I even changed the hosts file back).The cloudera-scm-server service is running, but am getting a couple errors when I check the status.cloudera-scm-server.service - Cloudera CM Server ServiceLoaded: loaded (/usr/lib/systemd/system/cloudera-scm-server.service; enabled; preset: disabled)Active: active (running) since Tue 2025-07-01 17:46:47 CDT; 23s agoProcess: 5922 ExecStartPre=/opt/cloudera/cm/bin/cm-server-pre (code=exited, status=0/SUCCESS)Main PID: 5924 (java)Tasks: 87 (limit: 819655)Memory: 986.0MCPU: 40.080sCGroup: /system.slice/cloudera-scm-server.service└─5924 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.432.b06-2.el9.x86_64/bin/java -cp ".:/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:lib/*" -server -Dlog4j.con

Hello Folks,I'd like to looking for your help on this.I seeking your guidance regarding how to join 2 JoinEnrichment Processors.Please help!!

Hello, I'm sending nifi tookit 1.27.0 nifi set-param commands to a nifi 1.27.0 clusterto add parameter key/values to an existing parameter contextIn an intermittent way,the nifi set-param commands properly set the key/value in the PCbut end in error with messageError deleting parameter context update request: Could not find a Request with identifier xxxI sometimes, not systematically, get the same error message when manipulating the PC key/value pairs via th UI ( when applying the changes ).From a network point of view I see 404 not found errors when the problem occursIs this a known issue ?Is there any way to avoid/solve this problem ?Regards

Hi all,I've been working with NiFi and observed an issue with the MergeContent processor. After running several experiments, I noticed that regardless of how many FlowFiles I send in, only the first one that enters the processor is included in the merged output. The rest seem to be ignored.My use case involves merging two FlowFiles and passing them into a custom Python script. However, since only one file is getting through, the processing is incomplete.I’ve verified that the incoming queue has both FlowFiles and that the processor settings appear standard. Has anyone encountered this?Could it be related to Merge Strategy or correlation attribute configuration?Are there specific settings required to ensure multiple FlowFiles are merged?Any guidance or suggestions would be appreciated. Thanks!