Solved

Does anyone know where to find the Maven repository for HDP 3.1.5.0-152?I want to compile HDP 3.1.5.0-152, but I can't find the corresponding Maven repository.At https://repo.hortonworks.com/content/repositories/releases/, only dependencies for other HDP versions are available.Thanks in advance.

Hi all,I've created a custom service in Cloudera Manager (ABC_ROLE) and defined a startRunner that calls a control.sh script. This script eventually runs a start_and_monitor function that starts two internal services and enters a monitoring loop.service.sdl Snippet:"roles": [ { "name": "ABC_ROLE", "startRunner": { "program": "scripts/control.sh", "args": ["start"], "environmentVariables": { "LOG_LEVEL": "${log_level}" } }, "stopRunner": { "relevantRoleTypes": ["ABC_ROLE"], "runner": { "program": "scripts/control.sh", "args": ["stop"] } } }],"rollingRestart": { "workerSteps": { "roleName": "ABC_ROLE", "bringDownCommands": ["Stop"], "bringUpCommands": ["Start"] }}control.sh Snippet:control.shstart_service() { echo "start_service() called" sleep 60 #Adding sleep to test exec ${SERVICE_PARCEL_HOME}/service/scripts/service_daemon.sh start_and_monitor}if [ "$CMD" == "start" ]; then start_serv

Hi All,I am trying to secure NIFI with Keycloak based access control. I am able to set up the admin account and login successfully. Next step, I want to create a group access policy that will have most of the admin privileges, however not via UI.  Is it possible to provide custom-authorizations.xml  and custom-users.xml on NIFI Helm where i specify my group and policy for it and still keep the initial admin user being generated as well? So basically, the content of these custom files should be appended to the generated users.xml and authorizations.xml file. I tried couple of tricks and hacks such as mounting my custom file under the name users.xml and authorizations.xml , this will basically replace auto-generated users.xml and authorizations.xml , hence it does not include admin user but only the group/users I mentioned. I tried to manually add my admin user to the file as well but in this case, root process-group uuid needs to be specified in the po

We have installed Cloudera Manager 7.13.1 and CDP Private Cloud Base 7.3.1 on Red Hat Enterprise Linux 9.4.However, we are unable to find a supported Spark 3 parcel that is compatible with this combination. We have reviewed the Cloudera Support Matrix but could not find any officially supported Spark 3 version for CDP Private Cloud Base 7.3.1.We attempted to use the following Spark3 parcels:SPARK3-3.3.2.3.3.7190.0-91-1.p0.45265883SPARK3-3.3.0.3.3.7180.0-274-1.p0.31212967Displays the following error when unpacking spark3 parcelsThere was an error communicating with the server:CDH 7.3.1 replaces SPARK3 3.3.2.3.3.7190.0-91-1.p0.45265883.

We’ve set up Cloudera Manager 7.13.1 along with CDP Private Cloud Base 7.3.1 on RHEL 9.4. As part of our environment validation, we’re looking to include Apache Spark 3 in the stack. However, we haven’t been able to locate a supported Spark 3 parcel that’s officially compatible with this combination.We’ve reviewed the Cloudera Support Matrix, but couldn’t find a clear reference for Spark 3 support with these specific versions of CDP and CM.According to a recent blog post,  Announcing availability of Spark 3.5 on Cloudera Unified Runtime which also directs the reader to What's New in Apache Spark (7.3.1). Apparently Spark 3 is supported by 7.3.1 and Spark 3.5 by 7.3.1 SP1.Does Spark still get installed separately or is it included with the base runtime?Should the support matrix be updated to reflect support for Spark 3? 

i am installing cloudera cdh  6.3.3 with isilon but when adding yarn i am facing this error  any help i will be grateful

I try to deploy and configure a CDP cluster over the CM RestAPI. However, I'm running into different problems around this. Starting with where to find values, e.g. /clusters/{clusterName}/parcels/products/{product}/versions/{version} where to find {product} and {version} for that.Or forcurl -X POST -u admin:admin -i -H "Content-Type: application/json" -d '{ "items" : [ { "ipAddress" : "{ipAddress1}", "hostname" : "{hostname1}" }, { "ipAddress" : "{ipAddress2}", "hostname" : "{hostname2}" } ] }' http://{cmIpAddress}:7180/api/v54/clusters/{clusterName}/hostsit returns following error although the Rest API documentation claims, that ipAddress is a required attributeHTTP/1.1 400 Bad Request[...]{"message" : "Unrecognized property: 'ipAddress'"}If I use the API Documentation on the CM it also does not fill out the information automatically, which would have been really helpful. So my question: could you provide the steps how to deploy and configure a CDP cluster with the CM Rest A

Hello,After upgrading CDP Private Cloud Base**from 7.1.7-1.cdh7.1.7.p2000.37147774 to 7.1.9-1.cdh7.1.9.p1013.58158856**,we can no longer access Hive databases through Hue behind Knox(Authentication backend: `KnoxSpnegoDjangoBackend### Symptoms1. Hue UI displaysSnippet type 1 is not configured.`).2. Hive databases and tables are no longer listed.3. Hue logs (`runcpserver.log`) showTypeError: Object of type WSGIRequest is not JSON serializable### Context* Hue is managed by Cloudera Manager; authentication is Knox → LDAP (Active Directory).* Hive is accessed through Knox (HTTPS 8443).* OS: RHEL 8.6; Python 3.8 shipped with the parcel.**Knox gateway log** records the following cookie-related error(ResponseProcessCookies.java:processCookies(130)) - Invalid cookie header:"Set-Cookie: csrftoken=1pPR9AOs9sRQaRGMxEws09bexNRMYg7W8SgJZGAlUzkBHZInYBs3IOaLBFisJjZj;expires=Tue, 12 May 2026 09:13:19 GMT; HttpOnly; Max-Age=31449600; Path=/;SameSite=Lax; Secure". Invalid 'expires' attribute: Tue, 12 Ma

Hi Sir,I am using apache nifi-2.0.0-M1 version. I am using window 10. I have some work. I need to export these task & import to production. But create template is missing.Here is image of process group:Process group configuration:How to export process group & import to other apache nifi using same version?  

 Hello Folks,I want to combine 3 FlowFiles using a UNION statement. I’ve already transformed the 3 FlowFiles to have the same schema and would like to merge them into a single content, structured like this:FlowFile1 ContentUNION ALLFlowFile2 ContentUNION ALLFlowFile3 ContentI’d appreciate your help with this.

Hi , My requirement is to consume all records from kafka topic (e.g. 1 lakh) in one go by using ConsumeKafkaRecord processor which is scheduled to run once 4 hours as i need to merge all data and create a parquet file and store in HDFS path using putHDFS processor .if i consume all incoming data and then use a merge content processor and run it once in 4 hours then issue comes is all data stays in nifi queued for 4 hours taking memory , so for avoiding that i wanted to use ConsumeKafkaRecord with scheduler (4 hrs) but its not able to consume all record .How to solve this issue ?

hello cloudera community Is it possible to know the date and time that a kudu table was accessed/read?i know there is "transient_lastDdlTime" field in kudu table but that field is updated only when the table is updated.

Hi, I have a fresh install of Cloudera Manager and it was running fine until I rebooted the server. The only thing I changed was the hosts file, but I don't think that has anything to do with it (I even changed the hosts file back).The cloudera-scm-server service is running, but am getting a couple errors when I check the status.cloudera-scm-server.service - Cloudera CM Server ServiceLoaded: loaded (/usr/lib/systemd/system/cloudera-scm-server.service; enabled; preset: disabled)Active: active (running) since Tue 2025-07-01 17:46:47 CDT; 23s agoProcess: 5922 ExecStartPre=/opt/cloudera/cm/bin/cm-server-pre (code=exited, status=0/SUCCESS)Main PID: 5924 (java)Tasks: 87 (limit: 819655)Memory: 986.0MCPU: 40.080sCGroup: /system.slice/cloudera-scm-server.service└─5924 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.432.b06-2.el9.x86_64/bin/java -cp ".:/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:lib/*" -server -Dlog4j.con

Hello Folks,I'd like to looking for your help on this.I seeking your guidance regarding how to join 2 JoinEnrichment Processors.Please help!!

Hello, I'm sending nifi tookit 1.27.0 nifi set-param commands to a nifi 1.27.0 clusterto add parameter key/values to an existing parameter contextIn an intermittent way,the nifi set-param commands properly set the key/value in the PCbut end in error with messageError deleting parameter context update request: Could not find a Request with identifier xxxI sometimes, not systematically, get the same error message when manipulating the PC key/value pairs via th UI ( when applying the changes ).From a network point of view I see 404 not found errors when the problem occursIs this a known issue ?Is there any way to avoid/solve this problem ?Regards

I can stop all role instances...curl -v -k -u me:$pass -X POST "https://host.domain:7183/api/v54/clusters/cluster1/services/NIFI/roleCommands/stop" -H "accept: application/json" -H "Content-Type: application/json" -d "{ \"items\": [ \"NIFI_NODE1\", \"NIFI_NODE2\" ]}"...but not the role on NIFI_NODE1 only. I am trying to write a script which detects that a node has disconnected, stops the role on that host, moves the flow.xml and flow.json and then restarts the role on that host.Is there anything that lets me stop and start a role on a single host?

https://medium.com/@surajnagendra/merge-csv-files-apache-nifi-21ba44e1b719I tried this approach but it didn't worked well

This video (timestamped) shows you can list the queue on connections, and see provenance and lineage in flow designer: https://youtu.be/8cZJ9CyLYyI?t=5904 But in the public cloud version of Cloudera Data Flow, that functionality is missing. I can list queue and see data in many formats, but no provenance and lineage. Do we need Data Hub to do this or am I missing something? I am doing this from the Web console of Data Flow Designer.I am trying this out on a trial account of the Cloudera DataFlow, so not sure if that affects my experience. Provenance and replay was an important function I wanted to try.  Screenshots (Imgur link) 

SERVICE_MONITOR_AGGREGATION_RUN_DURATIONThe health test result for SERVICE_MONITOR_AGGREGATION_RUN_DURATION has become bad: The last metrics aggregation run duration is 33.9 second(s). Critical threshold: 30 second(s).   Service Monitor File /var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR.log.out8:35:51.569 PM WARN Groups Potential performance problem: getGroups(user=hue) took 8024 milliseconds.8:36:21.270 PM WARN JvmPauseMonitor Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 18430ms: no GCs detected.8:36:21.558 PM WARN EnterpriseService com.cloudera.cmf.PollingScmProxy: run duration exceeded desired period. Duration: 19230 ms. Desired period: 1000 ms.8:36:22.253 PM INFO AggregatingTimeSeriesStore Run took PT33.916S which is over the slow run threshold of PT30S. 15690 metrics written for 28 entities. PT27.952S write time over 2 writes. Longest writes: PT27.876S,PT0.076S.8:37:02.720 PM INFO LDBPartition

Hi,I would like to know how can i get if it is available the company docker registry. To push the image to my company Docker registry.Best wishes,Salim 

Hello,We have noticed that Ranger policies are no longer being enforced in HDFS and Hive in our CDP 7.3.1 cluster. Users are able to access data despite explicit deny or allow rules configured in the Ranger UI.Details:Ranger UI is up and running; all policies are visible and enabled.HDFS and Hive services do not reflect the expected access restrictions.Restarting Ranger, Hive, and HDFS services did not resolve the issue.We are not sure when the issue started.Could you please assist us in identifying why the Ranger policies are no longer being enforced in HDFS and Hive? Let us know what additional logs or configurations would help with troubleshooting.

Hi guys,i hope you are doing well.can you please help me to get the root access in the container of the session in CDWS?best wishes,salim 

Hi,All applications within our portfolio utilize PgBouncer as the database connection pool mechanism, and consequently, any application-level pooling is to be disabled. Apache NiFi is one of the applications required to adhere to this standard. We are seeking guidance on the options available to disable database connection pooling in NiFi. Could you please advise on how we can accomplish this? Our current nifi version is 2.1Thanks for your help and let me know if more details are required. 

I am having an issue using the ExecuteSQLRecord processor and writing to JSON. For some reason the results returns a NULL value for one of my fields in the FlowFile results. However, if I execute the query directly in SQL Server, I get all my results with no NULL fields. I checked and there are no special characters, single quotes are escaped and nothing looks to be out of the ordinary. SQL Select Statement:Select emailTo = REPLACE(e.Email_Recipients, ';', ','), emailFrom = e.Email_Sender, emailSubject = e.Email_Subject, emailMessage = CASE WHEN c.ValidationCount = 0 THENREPLACE(e.Email_Message, '{0}','${filename}') WHEN c.ValidationCount = 1 THENCONCAT(REPLACE(e.Email_Message, '{0}','${filename}'), '<p>There is 1 validation error in this file</p>', v.Validations)ELSECONCAT(REPLACE(e.Email_Message, '{0}','${filename}'), '<p>There are ', c.ValidationCount, ' validation errors in this file</p>', v.Validations)END,    v.

Hello community,I have a PowerBi application with some dashboards. The dashboards are updated according to the configuration at least 2 times a day. The dashboards connected to Impala sometimes do not update and I get this error (see below). It does not happen all the time but many times. I have to update it manually every time I get the error. I use cloudera ODBC driver (64 bit) and basic gateway (PowerBi Cloud): on-premise data gateway.Can you help me to solve the update problems?Thank you,Translated with DeepL.com (free version)Error message:  {"error":{"code":"DM_GWPipeline_Gateway_MashupDataAccessError","pbi.error":{"code":"DM_GWPipeline_Gateway_MashupDataAccessError","parameters":{},"details":[{"code":"DM_ErrorDetailNameCode_UnderlyingErrorCode","detail":{"type":1,"value":"-2147467259"}}, {"code":"DM_ErrorDetailNameCode_UnderlyingErrorMessage","detail":{"type":1,"value":"ODBC: ERROR [HY000] [Microsoft][DriverSupport] (1170) Unexpected response received from server. Plea