Q&A

When the wizard generates credentials, it reports Insufficient access (50) ldap error, like this:/opt/cloudera/cm/bin/gen_credentials_ad.sh failed with exit code 50 and output of <<+ export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/usr/lib/mit/bin:/usr/bin+ KEYTAB_OUT=/var/run/cloudera-scm-server/cmf3782202571582054951.keytab+ PRINC=HTTP/cradle3302t.priv.cwru.edu@ADS.CASE.EDU+ USER=PruWGPfsVZ+ PASSWD=REDACTED+ DELETE_ON_REGENERATE=true+ SET_ENCRYPTION_TYPES=false+ ENC_TYPES_MASK=4+ USERACCOUNTCONTROL=66048+ ACCOUNTEXPIRES=0+ OBJECTCLASSES='objectClass: topobjectClass: personobjectClass: organizationalPersonobjectClass: user'+ EXTRA_ATTRIBUTES=+ DIST_NAME='CN=PruWGPfsVZ,OU=cradle33,OU=Hadoop,OU=Research Computing,OU=Information Technology Services,OU=Delegate

Hi,I am trying to install Atlas using (solr, Hbase, and Kafka), the issue when trying to start Atlas, I got the below logs and Atlas web UI give the below errorNote:- cluster kerberized and SSL EnabledHTTP ERROR 503 Service UnavailableURI:STATUS:MESSAGE:SERVLET:/503Service Unavailable-2024-06-15 07:07:08,377 INFO - [main:] ~ Cluster at utility01.swntq.corp:2181,master01.swntq.corp:2181,master02.swntq.corp:2181/solr-infra ready (ZkClientClusterStateProvider:178)2024-06-15 07:07:09,265 INFO - [main:] ~ Failed to obtain graph instance, retrying 3 times, error: java.lang.NullPointerException (AtlasGraphProvider:100)2024-06-15 07:08:08,550 WARN - [Thread-23:] ~ TGT renewal thread has been interrupted and will exit. (Login$1:203)2024-06-15 07:08:08,550 INFO - [ReadOnlyZKClient-utility01.swntq.corp:2181,master01.swntq.corp:2181,master02.swntq.corp:2181@0x076b05c0:] ~ Session: 0x2001da1e4992ffd closed (ZooKeeper:1288)2024-06-15 07:08:08,550 INFO - [ReadOnlyZKClient-utility01.swntq.corp:2181,ma

I am getting client can not communicate via ( token, kerberos) exception when I am trying to access yarn resource manager from java application. As part of my requirement in my spark java application I need to check some other applications running or not. For this I am using YarnClient class when I try to access this method getting above mentioned exception. Same working for me in my local system with kerberos authentication. I am using Hadoop library classes only to authenticate. UsergroupInformation class    

Hi AllSo glad that i have joined the Community.please can someone help me, i am an employee of a company that don't use cloudera (CDP) and i want to get the following certification : CDP-4001 or CDP-3002, with my own money.how can i access the CDP for practicing , i have heard that i can deploy it on the public cloud plateform (GCP, AWS and AZUR) , but i cannot see anything in the cloudera website about "create your cluster" or something like this.how do i access the CDP, or gets a CDP account ?

Hello.We are running CDP 7.1.7 and we have warning concerning 'Heap Dump Directory Free Space' :This role's Heap Dump Directory is on a filesystem with less than 5.0 GiB of its space free. /tmp (free: 3.7 GiB (94.74%), capacity: 3.9 GiB)Our filesystem is small :Filesystem Size Used Avail Use% Mounted on/dev/mapper/vg_sys-lvtmp 3.9G 8.5M 3.7G 1% /tmpIs it safe to reduce the threshold ?Currently, Warning is set to 5 GiB and Critical is set to 3 GiB .Or is it better / safer to increase /tmp filesystem size ?Because /tmp is also used for storing Event server and Host monitor data and when are also facing free space warning message .Regards

Hi everyone,We have a NiFi cluster with 3 nodes that was functioning fine until we encountered the following error. The cluster uses an embedded ZooKeeper for coordination. The error logs indicate issues with connection loss and leadership. Here are the relevant log entries:2024-06-19 16:25:05,335 WARN [Process Cluster Protocol Request-25] o.a.n.c.p.impl.SocketProtocolListener Failed processing protocol message from nifi01 due to org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling protocol message in response to message type: HEARTBEAT due to java.net.SocketException: Relais brisé (pipe) (Write failed)org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling protocol message in response to message type: HEARTBEAT due to java.net.SocketException: Relais brisé (pipe) (Write failed)        at org.apache.nifi.cluster.protocol.impl.SocketProtocolListener.dispatchRequest(SocketProtocolListener.java:186)   &nbsp

Hi,I am trying to integrate Nifi with slack by using putSlack processor.While doing so i have configured webhook url and webhook text. But while running i got PKIX certificate validation.So i have added slack cert in keyStore and added SSL context service property like below in putSlack.This time i didn't get certificate exception. But process failed with below error:Faile to parse JSON.Javax.json.stream.jsonparsingexception: Unexpected char 83.Unable to find any details regarding this issue. Could anyone please help.

I know I can run 'hbase version' in command line to check the HBase Version.Where can I find the version of hbase from Cloudera Manager?

Hi everyone,I'm new to NiFi and I'm trying to ingest data from more than 20 tables in an Oracle database into Cassandra. I have a few questions regarding the process:Can I transfer data from all the tables at once, without having to process each table individually? If so, what is the best approach to achieve this in NiFi?Is there a method to automatically create tables in Cassandra using Avro schemas within the NiFi flow? This is particularly important because in some use cases, we need to overwrite the table instead of appending data. How can we handle this scenario efficiently?

Hi AllWe are receiving data in JSON format (Nested and optional Nodes). I have written a JOLT to transform/flatten this JSON but not able to get desired result.I need help to fix JOLT to get desired output.Input JSON - Node - PartnerFunction and Indicator are optional. { "CustomerMaster": { "Rootnode": { "KUNNR": "0000028790", "NAME1": "NV Solucious", "LAND1": "BE", "SalesArea": [ { "VKORG": "0200", "VTWEG": "00", "SPART": "00" }, { "VKORG": "4932", "VTWEG": "00", "SPART": "00", "AUFSD1": "FT", "PartnerFunction": [ { "PARVW": "ZP", "PARZA": "000", "KUNN2": "0000028790" }, { "PARVW": "ZS", "PARZA": "000", "KUNN2": "90010523" } ] }, { "VKORG": "4932", "VTWEG": "10", "SPART": "00", "Indicato

Hi ,In my profile, I could see company as "London Stock Exchange Group PLC" and it needs to be change to "Refinitiv US LLC". Can you please refer my teammate details (Srikanth Macha) who is in same company and do the needful.Thanks, Lakshmi

Hello,I have this error when I connect a websocket client (tried 3 different ones) :2024-06-12 19:28:10,731 WARN [qtp293746312-1015] o.a.n.w.jetty.RoutingWebSocketListener Unhandled Error: Endpoint org.apache.nifi.websocket.jetty.RoutingWebSocketListener : org.eclipse.jetty.websocket.api.exceptions.CloseException: org.eclipse.jetty.websocket.core.exception.WebSocketException: RoutingWebSocketListener OPEN method error: nullI see connection successful on the client and then it gets disconnected immediately.Client and server are on the same machine, url is ws://127.0.0.1:8000/test (or any hostname, port and path). I suspected a firewall issue but Python's websockets server is fine.Thanks for your time.

Application application_1655291383854_84179 failed 2 times due to AM Container for appattempt_1655291383854_84179_000002 exited with exitCode: 1 Failing this attempt.Diagnostics: [2022-06-17 00:49:32.617]Exception from container-launch. Container id: container_e1071_1655291383854_84179_02_000001 Exit code: 1 Exception message: Launch container failed Shell output: main : command provided 1 main : run as user is sapproep main : requested yarn user is sapproep Getting exit code file... Creating script paths... Writing pid file... Writing to tmp file /data/data02/yarn/nm/nmPrivate/application_1655291383854_84179/container_e1071_1655291383854_84179_02_000001/container_e1071_1655291383854_84179_02_000001.pid.tmp Writing to cgroup task files... Creating local dirs... Launching container... [2022-06-17 00:49:32.619]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : Error un-registering AM client java.lang.Nul

We have a source directory for Getfile which has one thousand subdirectories (i.e. there are one thousand users who each have a Windows share).Processing issues arise when a user drops several thousand files into their directory.  I presume Getfile scans each directory sequentially and when it finds files, it empties the directory (we delete the source file).  So when it comes across a directory that has several thousand files or a directory that is being constantly written to, that user effectively shuts out everyone for several minutes or tens of minutes.What I would like to do when coming across a directory with many files is to pick up say one hundred and then move on to the next directory.  This would allow for a more even distribution among users.There is <path> attribute that distinguishes between users but I'm not sure how to take advantage of that to solve my problem.  Thanks in advance for any tips.

Hello, I want to migrate data from MsSQL Server to Oracle Database. I added ConnectionPools for both of. I am trying to export data andimport data ,but i cant. Which processor i need to use and how ? I tried to fecth data with ExecuteSQL but it is fetching as Avro. Than i wanted to convert it to json but i didn't. Because i cant configure the ConvertRecord processor.

On our Data Flow Environment, we are getting alert " Metering Application Unavailable", wanted to understand what this indicates  and a way to fix this.Thanks 

hello cloudera community, could you help us with a problem regarding the /var/log directory at certain times (we couldn't identify the right moment) the /var/log directory is changing the permissions to 750 and with that the CDH 6.3.x cluster services is failing because it can't write the logs. Have you ever had a problem like this? did you manage to solve it? we had to leave a crontab running every 5 minutes to always change it to 755. ps: selinux is disabledps: firewall is disabled

Hi,  i want to ask about nifi login, i can login but cant acces nifi ui in nifi-user log success. this my nifi-user.log and error in nifi ui  thank you 

We have a cloudera cluster version 3.1, fully kerberized. We have ranger with HDFS, Hive, HBASE, among other plugins active. All of the sudden, the HBASE service gets stopped after saying that the Ambari Metrics user does not have permissions under /apps/hbase/*. We have the ranger policy in HDFS where the user has read, write, execute permissions in the mentioned paths. Yet we see in the Ranger audit page the denial, but after HDP ACL rule, not Ranger rule. I did the plugin deactivation and activation, I checked as well the plugins sync time, and the rules in place @ the hbase server servers (all looks ok) We have another cluster with same configuration, that is working perfectly fine. Any idea, where to look at? I have the feeling that the problem is coming with Kerberos, but I don´t see any "evident" issue. Thanks!

Hi,I want to put the alert name in the subject of the email alert I followed the example in https://github.com/apache/ambari/blob/branch-2.1/ambari-server/src/main/resources/alert-templates.xml but the following does not fill in the  variables and I just the the variable names in the email.<![CDATA[Ambari: [$alert.getAlertState()]-[$alert.getAlertName()]-[$alert.getAlertText()]]]>If I use the template as it is, the email populates with the summary variables, ie:<![CDATA[Alert Summary: OK[$summary.getOkCount()], Warning[$summary.getWarningCount()], Critical[$summary.getCriticalCount()], Unknown[$summary.getUnknownCount()]]]>Is there a trick to get the subject to show the $alert variables to the subject?Thanks in advance! Eric 

We are facing no applicable policies issue on nifi ui ,then we renew Kerberos tickit it works ,but after that we got know that cluster status is disconnectedAnd git below popup.       Please help in this, nifi-app.log not showing any errorexcept one warn, failed to send heartbeat to nifi.cluster.protocal.protocalexception, PKIX validation failed 

Hi,I am in need for some sort of alerting mechanism to monitor long running CDE jobs, basically we use Airflow to call /run Spark DBT jobs I am looking if there is a way to monitor / alert if any jobs cross certain threshold time limit.Would appreciate if anyone has implemented it / guide in this direction.Thanks Wert

While executing the spark application in yarn i was facing the following issue twice or thrice a day these days. ERROR MicroBatchExecution:91 - Query [id = 0ec965b7-5dda-43be-940c-3ec8672bcd5c, runId = 17c15719-ab20-4488-ba37-ccf6a6ca27e1] terminated with errororg.apache.spark.SparkException: Job aborted due to stage failure: ResultStage 185575 (start at DeviceLocationDataListener.scala:148) has failed the maximum allowable number of times: 4. Most recent failure reason: org.apache.spark.shuffle.FetchFailedException: Stream is corruptedat org.apache.spark.storage.ShuffleBlockFetcherIterator.throwFetchFailedException(ShuffleBlockFetcherIterator.scala:554)at org.apache.spark.storage.ShuffleBlockFetcherIterator.next(ShuffleBlockFetcherIterator.scala:470)at org.apache.spark.storage.ShuffleBlockFetcherIterator.next(ShuffleBlockFetcherIterator.scala:64)at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:435) at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:441)at sc

I have 2 cluster say A,B. Data is injecting to A & Replication is enabled from A to B.If A is down and data injection is started to B.  A came up after some hours. Is it possible to transfer only the delta data & future data from B to A using the cloudera replication manager?

Hello,Complete noob here.I have a file, and have created an iceberg table over it (via Hue). However, I can't select any data. How do I set the schema up correctly?CREATE TABLE iceberg_test_1 (month STRING, day STRING, time STRING, host STRING, source STRING, body STRING) PARTITIONED BY(event_day INT, event_month INT, event_year INT) STORED BY ICEBERG LOCATION 'ofs://ozone/user/logs/test';The file is:Jun 4 08:26:57 host smartd[2657]: Device: /dev/bus/0 [megaraid_disk_05], SMART Failure: DATA CHANNEL IMPENDING FAILURE GENERAL HARD DRIVE FAILURE