Solved

Hi all,I am trying to connect DBeaver from my windows 10 client PC to my CDP Hive cluster, but failing with an error message of following Caused by: com.cloudera.hiveserver2.support.exceptions.GeneralException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: [Cloudera][HiveJDBCDriver](500169) Unable to connect to server: GSS initiate failed.My setup CDP Version : 7.1.9 CDP OS : RHEL 9.1MIT Kerberos Version : 4.1DBeaver Version : 24.3.4.202502021521KDC : Microsoft Active Directory dbeaver.ini -startupplugins/org.jkiss.dbeaver.launcher_1.0.22.202502021521.jar--launcher.libraryplugins/org.eclipse.equinox.launcher.win32.win32.x86_64_1.2.1100.v20240722-2106-vmargs-XX:+IgnoreUnrecognizedVMOptions-Dosgi.requiredJavaVersion=17-Dfile.encoding=UTF-8--add-modules=ALL-SYSTEM--add-opens=java.base/java.io=ALL-UNNAMED--add-opens=java.base/java.lang=ALL-UNNAMED--add-opens=java.base/java.lang.reflect=ALL-UNNAMED--add-opens=java

/etc/cloudera-scm-agent/config.ini[General]server_host=192.168.88.54server_port=7182max_collection_wait_seconds=10.0metrics_url_timeout_seconds=30.0task_metrics_timeout_seconds=5.0monitored_nodev_filesystem_types=nfs,nfs4,tmpfslocal_filesystem_whitelist=ext2,ext3,ext4,xfsimpala_profile_bundle_max_bytes=1073741824stacks_log_bundle_max_bytes=1073741824stacks_log_max_uncompressed_file_size_bytes=5242880orphan_process_dir_staleness_threshold=5184000orphan_process_dir_refresh_interval=3600scm_debug=INFOdns_resolution_collection_interval_seconds=60dns_resolution_collection_timeout_seconds=30[Security]use_tls=0max_cert_depth=9[Hadoop][Cloudera][JDBC][Cgroup_Paths]The Internet said that the problem of auto-tls, but I did not open, try a lot of online methods can not solve, can anyone help me solve this problem

Hi @MattWho Need your help and suggestion here as i went through many of the Nifi related article here , since my use case is different need your valuable suggestion .1. i want to know how to use SSL\TLS to have https mode for nifi url 2. in our organization already we are generating signed certificate with SAN entries asnifi-dev.x.x.net [domain we try to access via browser]nifinode1.x.x.netnifinode2.x.x.netnifinode3.x.x.netzookeepernode1.x.x.netzookeepernode2.x.x.netzookeepernode3.x.x.netpost request raised we get a file nifi.crt file with this what else certificates are required\Needed?3. as trial basis tried already creating keystore and truststore but seems some issue 4. 5. 

Hi Everyone,I’m Sachin Duggal, and I manage a bag manufacturing business. I’m currently exploring HDFS data recovery methods and have come across a question: Is it possible to recover HDFS data using only the block data? If you’ve dealt with something similar or have any suggestions on how this could be done, I’d love to hear your thoughts.Thank you in advance,Sachin Duggal

HI @MattWho Good Day, we are experiencing new error called"  Connection refused: zookeepernode2" post setting up DNS  with the keystore and Password for NIFI which we generated internally using keytool, same keystore and password we are using in zookeeper as well.Attaching the Required details, Request you to please check and provide your valuable feedbackBelow is the web UI error. we setted up the nifi cluster in below formatNIFINODE1, NIFINODE2. two azure vm'sZookeepernode1, Zookeepernode2, Zookeepernode3. three Azure vm'splease find zoo,cfg details below.#zookeeper node3 has been choosen has an electorsudo bin/zkServer.sh status/usr/bin/javaZooKeeper JMX enabled by defaultUsing config: /opt/zookeeper/bin/../conf/zoo.cfgClient port not found in static config file. Looking in dynamic config file.grep: : No such file or directoryClient port not found in the server configsClient port found: 2181. Client address: localhost. Client SSL: false.Mode: leader____

Hi everyoneWhen installing cloudera-scm-agent, the installation failed due to heartbeat check failure. However, the backend logs show no errors and even output INFO-level logs indicating successful heartbeat detection. What could be the root cause?agent.loginstalling logserver.log

Hi,I'm looking for a way to implement a process group level counter that I can read and update either with a processor, including ExecuteScript / ExecuteGroovyScript.I've looked at and tried:1. Counters: I can create one, set the value and increment it, but not read the value within a processor, that I've found as of yet.2. Parameter Context parameters: I can read the value, but not increment or reset it. It seems to be static once it's defined.Is there something I've missed with the above, or does anyone have another suggestion please?Thank you!

Hi All,I have created my processor group in a local instance of NiFi 2.0.0-M4 and I have downloaded the json definition file and also I can able to import that json manually to a target nifi system (production), but my ultimate goal is to deploy this json using NiFi RestAPI and I have accomplished the following tasks1. Generate the NiFi Token2. Create a ProcessorGroup to the target NiFi systemBut I could not find a way to import the json definition file through REST API and when I search on the apache NiFi documentation of API and in could not find any proper api call for NiFi version 2.0.0-M4 but there is only for version 1.28 supported Api endpoints for template concepts but as of NiFi version 2.0.0-M4 it was changed from xml template into json definition concept. Now how do I import the json file to the target system processGroup which i created through rest api.Please anyone can give your suggestion asap.ThanksVijay.AX

Does anyone here can share if we can perform stop and start of roles using commandline in Kafka backend if the Cloudera Manager is down? 

Hi,I'm still trying to get this to work:https://community.cloudera.com/t5/Support-Questions/Process-Group-level-counter/td-p/402648I have versions NiFi 2.0.0 and 2.2.0 at my disposal, and have tried to get simple examples working that I can then implement in my flow. I'm having difficulties as all the examples I find are with older versions of NiFi.First, Distributed Server and Client Services have been renamed:DistributedMapCacheServer has been renamed to MapCacheServerDistributedSetCacheServer has been renamed to SetCacheServerDistributedMapCacheClientService has been renamed to MapCacheClientService DistributedSetCacheClientService has been renamed to SetCacheClientServiceI've mostly gotten PutDistributedMapCache and FetchDistributedMapCache working as per the older example here:https://community.cloudera.com/t5/Support-Questions/how-to-put-data-in-PutDistributedMapCache/td-p/171803andhttps://www.youtube.com/watch?v=wjis28NrF90 In the Controller Settings under Management Controller

Hi,I have a use case where I have to read the JSON files from "/data/json_files/" location with schema enforced.For the completeness we want to mark the invalid records. The invalid records may be the ones where the mandatory field/s are null, data type mismatch or invalid json itself.I have tried below but nothing worked as of now. It would be nice if someone has already this use case and a solution for it or may be knowledgeable in this area. Example Schema:schema = StructType(    [        StructField(            "meta",            StructType(                [                    StructField("id", StringType(), False),                    StructField("timestamp", TimestampType(), False),                   

hola, estoy teniendo problemas con la descarga de CLOUDERA, alguien me puede dar las indicaciones por favor gracias

Hi All,This link has queries to get the custom utilization report -- https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/monitoring-and-diagnostics/topics/cm-cluster-utilization-report-metrics-queries.htmlIf I need 6 months data for each CPU utilization, memory utilization -- do we have a date column to specify the required date range.Where to execute these queries in CDSW ?.There are no table name specified in few queries. Will those queries fetch results ?Thank You.

Hello Everyone,I am part of the MSSP team, and we have received a request to support Cloudera Navigator. Based on the documentation, it appears to be part of the Cloudera Security Suite. However, I also came across other products like Atlas and Sentra.Could you please confirm if Navigator is still included in the Security Suite? Additionally, under which product does it fall, or is it available separately? Any specific details about Navigator, as well as insights into the suite or product structure, would be greatly appreciated.Thank you!

i have issue in connecting external zookeeper and nifi cluster , i have 3 nodes like zookeepernode1 zookeepernode2 zookeepernode3 and nifinode1 and nifinode2nifiuser@zookeepernode3:~$ /opt/zookeeper/bin/zkServer.sh status/usr/bin/javaZooKeeper JMX enabled by defaultUsing config: /opt/zookeeper/bin/../conf/zoo.cfgmyid could not be determined, will not able to locate clientPort in the server configs.Client port found: 2181. Client address: localhost. Client SSL: false.Mode: followerroot@zookeepernode1:~# /opt/zookeeper/bin/zkServer.sh statusZooKeeper JMX enabled by defaultUsing config: /opt/zookeeper/bin/../conf/zoo.cfgClient port not found in static config file. Looking in dynamic config file.grep: : No such file or directoryClient port not found in the server configsClient port found: 2181. Client address: localhost. Client SSL: false.Mode: followerroot@zookeepernode2:/opt/zookeeper/logs# /opt/zookeeper/bin/zkServer.sh statusZooKeeper JMX enabled by defaultUsing config: /opt/

Good day,There are NiFi processors that store state in memory, one example being the DetectDuplicate processor, which "remembers" FlowFiles based on a specific attribute. The duration for which this memory is retained is determined by the Age Off Duration parameter. I would like to know the default value for this parameter and whether it is possible to configure it to retain the memory indefinitely (i.e., with no expiration). Currently, I am using a value of '365000 days'.Additionally, I would like to know if there is a way to preserve the detected values even after a NiFi restart.

Hello everyone,I have a question regarding to data provenance. I did read NiFi document about provenance repository that can store the provenance data, but I want to this provenance data in database for long-term. How can I do that in NiFi 2.2.0?

I am running Ranger Admin 2.4.0. Right now I have it synced with LDAP and am able to login to the Ranger Admin UI via my LDAP username and password. When I try upgrading to Ranger Admin 2.5.0 using the same configurations I am getting these errors in the logs.catalina.out - java.lang.RuntimeException: Failed to create user drew.nicolette in x_portal_user table. retryingat org.apache.ranger.biz.XUserMgr$ExternalUserCreator.createExternalUser(XUserMgr.java:3314)at org.apache.ranger.biz.XUserMgr$ExternalUserCreator.run(XUserMgr.java:3288)at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.addRunnable(RangerTransactionSynchronizationAdapter.java:136)at org.apache.ranger.common.db.RangerTransactionSynchronizationAdapter.executeOnTransactionCommit(RangerTransactionSynchronizationAdapter.java:82)at org.apache.ranger.biz.XUserMgr.createServiceConfigUser(XUserMgr.java:2601)at org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke(<generated>)at org

HiMy one liner question:How can we ensure that the DatabaseTableSchemaRegistry recognizes changes in the table structure and updates the corresponding table schema definition?Details:Our PutDatabaseRecord processors' readers are configured to use the DatabaseTableSchemaRegistry to parse incoming JSON. We have observed multiple instances where the NiFi flow fails when the target table structure is altered, such as when columns are added or removed.When new columns are added to the target table, the PutDatabaseRecord reader does not recognize the change and continues to create insert statements without the new columns. We are unsure what may have resolved the issue, but in some instances, it was automatically resolved after a day. We suspect there may be an internal cache that is rebuilt periodically. We have no clear solution for this issue and would greatly appreciate any assistance regarding it.Our NiFi system is a 3-node cluster (version 2.1) that uses Kubernetes-based leader electio

Hi,I am trying to prepare a report listing Client IP addresses connecting to various Cloudera services. I know in Ranger Admin UI -> Audits -> Access, I can see this information. However, because of the huge list of records, it is not practical to copy each page from here. Hence, below are options i can think of to fetch the data:1. Ranger database: I am unsure which table in ranger DB stores this information from where Ranger Admin UI gets the data. 2. Solr: For our Ranger instance, source is configured as Solr. However, for some reason I am unable to query Solr via Solr UI. Possibly it could be coz of some permissions issues, as when attempted to get this data using Solr API, i get authentication error. Unsure where should I check for the permissions as in Solr Policy in Ranger, I have given all the reasonable permissions that I could think of to myself. Still no luck.Please suggestThankssnm1523

Hi,We are using Cloudera Private Cloud, and our customer wants to process SAS files using NiFi. However, I couldn't find a built-in NiFi processor that supports this.Is there a known solution or workaround for handling SAS files in NiFi? Any suggestions would be greatly appreciated.Thanks

Hi We are currently in the process of upgrading our NIFI version to 2.0.0. But as per the below NIFI jira task PrometheusReportingTask has been completely removed from version 2.0.0 and added as a NIFI rest api endpoint.https://issues.apache.org/jira/browse/NIFI-13507The issue we are currently facing is that with old version 2.0.0-M2 metrics was exposed through PrometheusReportingTask which can be accessed without the need for any authentication and we were able to scrape the metrics to Prometheus.But with the latest versions from 2.0.0 we have to provide the Bearer Token(Using OIDC) to access the metrics as it is a Rest API endpoint which is causing difficulties to integrate with Prometheus as we need to add an extra layer to get the token.So my question here is, If there is any way to access the nifi-api/flow/metrics/prometheus without the need to provide any authentication which will solve our issue.Any Suggestions are appreciated. 

Hello,in hive i am unable to establish a connection,beeline> !connect jdbc:hive2://localhost:10000/Connecting to jdbc:hive2://localhost:10000/Enter username for jdbc:hive2://localhost:10000/:I don't know what to put for username and password, it is not accepting ENTER Key or my device login details.Output: 24/05/20 01:19:08 [main]: WARN jdbc.HiveConnection: Failed to connect to localhost:10000Could not open connection to the HS2 server. Please check the server URI and if the URI is correct, then ask the administrator to check the server status. Enable verbose error messages (--verbose=true) for more information.Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10000/: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0)Please help.

I'm encountering challenges when using the InvokeHTTP processor in Apache NiFi to send GraphQL POST requests. Specifically, I'm trying to execute GraphQL mutations via POST but facing difficulties in configuring the InvokeHTTP processor to handle these requests properly.Problem:I've set up an InvokeHTTP processor to send GraphQL mutation requests via POST to a specific endpoint. However, the requests don't seem to be reaching the GraphQL server as expected. Details:The InvokeHTTP processor is configured with the following properties:HTTP Method: POSTRemote URL: myURL...!Headers: Content-Type: application/json, Accept: application/jsonSend Parameters in: BodyThe body content is constructed in JSON format as required by the GraphQL server. It includes the mutation query and variables. Request for Help:I'm seeking guidance on how to properly configure the InvokeHTTP processor in NiFi to successfully send GraphQL POST requests. If anyone has experience using InvokeHTTP for GraphQ

I am fairly new to Nifi and I am using 2 GetSFTP processors to get file from two remote servers, these are application.properties files. Now I want to compare the contents of these files and check for any differences.Need a little guidance on how I can achieve this in NiFi.