Find answers, ask questions, and share your expertise
Recently active
22/05/25 01:43:01 INFO utils.HiveUtils: load data into hive tableHive Session ID = fc864c41-2236-472d-ac57-96c6df6d8e4822/05/25 01:43:01 INFO SessionState: Hive Session ID = fc864c41-2236-472d-ac57-96c6df6d8e4822/05/25 01:43:01 INFO session.SessionState: Created HDFS directory: /tmp/hive/hive/fc864c41-2236-472d-ac57-96c6df6d8e4822/05/25 01:43:01 INFO session.SessionState: Created local directory: /tmp/root/fc864c41-2236-472d-ac57-96c6df6d8e4822/05/25 01:43:01 INFO session.SessionState: Created HDFS directory: /tmp/hive/hive/fc864c41-2236-472d-ac57-96c6df6d8e48/_tmp_space.db22/05/25 01:43:01 INFO tez.TezSessionState: User of session id fc864c41-2236-472d-ac57-96c6df6d8e48 is hive22/05/25 01:43:01 INFO tez.TezSessionState: Created new resources: null22/05/25 01:43:01 INFO tez.DagUtils: Jar dir is null / directory doesn't exist. Choosing HIVE_INSTALL_DIR - /user/hive/.hiveJars22/05/25 01:43:01 INFO tez.TezSessionState: Computed sha: c765910321290465015ccaaeae5dde480fd29e141a4c11594756ecfe
I was able to replicate my problem using the following configuration:FlowThe MergeContent configuration is as followsMergeContent PropertiesMergeContent SchedulingAlthough the conditions of the MergeContent processor are met, it does not concatenate the incoming flowfiles.The last version in which the above flow worked was version 1.15.3.Any suggestions on how to solve this? Is this a bug?P.S.: The flow is so designed due to a dependency on only writing one file per day.
Hi all,i'm searching for a safe procedure to add a new journal node to a Cloudera CDP cluster. Now we have only two master node and so we would like to add the third masternode and deploy the third journal node on it. Is it enough to deploy the role using the cloudera manager add role function? Or is it necessary to take further action? Thank you.
Hello,due to a problem with a script, we have almost saturated the HDFS available space.I suppose this is caused by temporary Hive files that have not been cleaned up due to the abnormal termination of the script.I would like to check the /tmp/hive folder on HDFS but my user, that has administrative privileges, cannot access the /tmp/hive folder.Is there a way to check and clean such folder? Any help would be appreciated. KRs,Andrea
hello, please help with below questions: 1. I have nifi cluster with 2 nodes and running ExecuteGroovyScript processor to create text formatted file generated from flow data content and save it on local filesystem but it is saving file only on one nifi node, is there any way to replicate this file on all nodes at a time? 2. Above file formatting by ExecuteGroovyScript processor as text in Prometheus format and using such flow http_request-> fetchfile-> http_response processors to create custom Prometheus healthcheckmetric web page, it worked for single node but with 2 nodes all data balanced randomly between 2 nodes so I have different content in 2 web pages ( http://node1/healthcheck/ and http://node2/healthcheck/ ) so monitoring not working properly, could you help how to add custom Prometheus metric so it will be available like regular nifi prometheus web page /metrics/ with same content on both nodes? may be other solutions?&n
Hi All, We are facing an issue while adding new host to Cloudera manger. During host installation, Cloudera manager is replacing the /etc/yum.repos.d/cloudera-manager.repo file. The original repo file on server has the required Cloudera credentials to downlaod packages from Cloudera archive. -------- $ cat cloudera-manager.repo [cloudera-manager] name = Cloudera Manager, Version 5.13.1 #baseurl = https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.13.1/ #gpgkey = https://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera baseurl = https://7bf0xxxx-xxxx-xxxxxxb236:15xxxxxxxxxc9@archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.13.1/ gpgkey = https://7bf0xxxx-xxxx-xxxxxxb236:15xxxxxxxxxc9@archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera gpgcheck = 1 -------- Now when we try to add this host in Cloudera manager, during installation phase the file Cloudera-manager.repo is replaced. Due to this we are observing authentication failures.
Ranger is not updating the access audits in ranger after a certa date solr logs are giving the below error:ERROR (qtp246273275-120) [c:ranger_audits s:shard1 r:core_node2 x:ranger_audits_shard1_replica_n1] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Exception writing document id 8cb43886-8056-46b2-8b24-0ead48c8280b-1 to the index; possible analysis error: number of documents in the index cannot exceed 2147483519 how can this be troubleshooted?
Hi Community, We are working on a new requirement where I need to disable table-level TTL and rather use cell-level TTL for one of our HBase tables. I can disable the table-level TTL by altering the table property and set cell-level TTL for new incoming data. However, I can't find anything on how can I set cell-level TTL for the data / rows that are already existing in the table. Any suggestions will be very helpful. Thank you,Snehasish
Hello,We are observing Impala CatalogException: Error refreshing metadata for table due to lock contention errors on our Cluster.Came across this article - https://community.cloudera.com/t5/Support-Questions/Impala-Error-updating-the-catalog-due-to-lock-contention/td-p/329310 upon checking the suggestions made by Cloudera Tech - load_catalog_in_background is already unchecked nor do we find any error for JVM Pause in Catalogue LogsJava Heap Size of Catalog Server in Bytes = 28GCDH 6.3.3Would appreciate any help in understanding why this happens and ways to fix this.
So I am trying to implement this script from datetime import datetime, timedelta, date flowfile = session.get()current_date= datetime.today().date()if flowfile != None: src_table_name = flowfile.getAttribute('src_table_name') date_filter = str(current_date.replace(day =11)) if src_table_name in ['Income_project_BP', 'Plan_final_scenario', 'Plan_primary_scenario', 'ProductionCalendar', 'Subcontracting_BP']: flowfile = session.putAttribute(flowfile, 'date_filter', date_filter) session.transfer(flowfile, REL_SUCCESS) else: session.transfer(flowfile, REL_FAILURE) And I've tried a lot of different ways but when it comes to "replace" method I'm always getting this error. Same thing when I am trying to use "strftime" method What should I do? I really need this method.Thank you beforehand!
Hello folks! I have a cluster with 9 machines, running on CDH 6.2 (OnPremise). I have 3 master, 1 edge and 5 workers. I am not able to up 2 of 5 NodeManager on workers. 3 of them are ok, and 2 of them give me a follow log (attach), without error but a Warning with "NullPointerException":. When I put the NodeManager to run, on Cloudera Manager it doens't fail, but I got two alerts, as follow:- NodeManager can not connect to ResourceManager- ResourceManager could not connect to Web Server of NodeManager Also, I can't access the /jmx of the server. And, when I run NodeManager by Cloudera Manager, my CPU going to use of 100%. On that 2 workers, I have RegionServer and DataNode working fine, the problem is only with NodeManager. Please, any suggest?
kudu service has 3 kudu master and 5 kudu tablet among these 5 kudu tablet, 3 of them have high memory usage, 60GB has been set for each kudu tablet and it is using more than 85% to 90% of that memory and with that it ends up causing jobs failure below shows the memory usage behavior:TABLET SERVER: hostname04.local - USE MEMORY: 55.28G / %: 92.13%TABLET SERVER: hostname05.local - USE MEMORY: 54.22G / %: 90.38%TABLET SERVER: hostname06.local - USE MEMORY: 29.05G / %: 48.42%TABLET SERVER: hostname07.local - USE MEMORY: 28.47G / %: 47.46%TABLET SERVER: hostname08.local - USE MEMORY: 54.99G / %: 91.66% memory usage graph of all kudu tablet in cloudera manager: would like to understand why kudu is having this behavior and how can we reduce this memory usage? PS: kudu service is installed and being managed by cloudera manager express in version 5.16.2 and kudu service is in version 1.7
Good morning everyone, I have a question regarding the Failure of the ExecuteSQL processor. I would like to know if when accusing a failure in this processor I can give a Retry in the Processor with its own Failure. Thanks
Working on a wait-notify setup, but it is running into issues. It is constantly being populated with an "Address already in use" error although there is just the one set of DistributedMapCacheServer and DistributedMapCacheClient controller services. This oddly doesn't immediately cause an issue when these are first created and the wait-notify actually works fine, but on any subsequent runs with the same input file (same Release Signal Identifier) the files do not wait at all and move through immediately.
Hi, I am facing an issue with one of our Impala Daemon which continuously has error related to “The health test result for IMPALAD_FRONTEND_CONNECTIONS has become bad: There are 0 (Beeswax pool) 123 (Hive Server 2 pool) active client connections, each pool has a configured maximum of 128” Have checked CM -> Impala -> Charts Library -> check "Active Frontend API connection" this shows too many Hive Server connections Also checked server's network side and run netstat -an | grep ESTABLISHED | grep 25000 but I see only 2 active connections. [root@nucleus ~]$ netstat -an | grep ESTABLISHED | grep 25000tcp 0 0 10.20.20.41:25000 10.20.20.41:49728 ESTABLISHED tcp 0 0 10.20.20.41:25000 10.20.20.41:49732 ESTABLISHED Any assistance / guidance in resolving this issue is appericated. CM / CDH - 6.3.3 Thanks Wert.
I am having 5 region servers in my cluster, I have added an additional 5 region servers,can I directly decommission the older 5 region server with the help of ambari server, will it cause any data loss in HBase?
Hello, Does Cloudera 6.3.3 (CM/CDH) support Oracle 19c ?
what version can be used as freeware like cloudera express in present CDP after merger as i see current cloudera express is discontinued If there is any freeware or open source still available to use and list of components please.
I have been trying to download HDP Sandbox for last 1 month but not able to complete it up-to 100%. Its size is 20 GB and I have utilized more than 150 GB still not able to complete this 20 GB download as if fails after some times. While every other providers provide torrent link for such huge file size I don't know why cloudera has not provided. And if you complain about my internet speed then FYI I tried downloading it from my Company network too. I am unable to digest that people at Cloudera can't think of this issue and provide some alternative way to download it. I tried searching for alternate way to download it but could not find it. Here is what you can do to solve this issue: 1. Provide a link that is resume-able. 2. Provide a Torrent link. No Thanks for wasting my time and bandwidth. Sam
Hi everyone!I'm trying to flat array and it view as simple case, but I can't reach the goal.Here is simple json input:{ "MainArray": { "mattr1": "some_value1", "mattr2": "some_value2", "Subarray": { "sattr1": 1, "sattr2": 2, "sattr3": 3 } }}Then I would like to convert it to array of arrays [key, value],Here is Desired Output:{ "ResultArray" : [ [ "mattr1", "some_value1" ], [ "mattr2", "some_value2" ], [ "sattr1", 1 ], [ "sattr2", 2 ], [ "sattr1", 3 ] ]} Here is my spec:[ { "operation": "shift", "spec": { "MainArray": { "*": { "$": "ResultArray[#2][0]", "@": "ResultArray[#2][1]" }, "Subarray": { "*": { "$": "ResultArray[#3][#2][0]", "@": "ResultArray[#3][#2][1]" } } } } }] And I don't understand how to fix problem when Subarrays is inside extra array in my result:{ "ResultArray
I wonder where the query history like the image above is saved to!Help ME! 😞
If we are going to decomm a data node on which a region server is also running : 1. first - is that a viable option? Can a data node be decommissioned when a region server is also running on that host? Doesn't the region server have/use data on that data node - if so, then how can the data node be decommissioned? 2. let us suppose to due some severed hardware error the node has to be decommed. Then is it necessary to shift the region server on to another node? Or (if there is no real issue with the hardware etc) can a region server exist on a machine without a data node on it? 3. Is there anything to be done to HBASE/Region Server (maintenance mode etc) when you decommission a data node? Appreciate the insights.
I am facing an issue while starting resource manager using ambari but able to start it using CLI.when starting using ambari no logs is generated in /var/log/haoop-yarn/ location.Please find logs on ambari. Traceback (most recent call last):File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 995, in restartself.status(env)File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/resourcemanager.py", line 150, in statuscheck_process_status(status_params.resourcemanager_pid_file)File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/check_process_status.py", line 43, in check_process_statusraise ComponentIsNotRunning()ComponentIsNotRunningThe above exception was the cause of the following exception:Traceback (most recent call last):File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/YARN/package/scripts/resourcemanager.py", line 261, in <module>Resourcemanager().execute()File "/usr/lib/ambari-agent/li
Hello, I am new here, need some help I Have CDP5 with 2 masters and 3 nodes, one of the masters is trying to get HBase DS tables and schemas, when doing from Workers all good but when trying to get from master it goes to localhost:2181 and not creating a job for Zookeeper workers, at least after 3 tries giving error of refused exception, I don't understand why master behave like that and why I cannot get data from Master, looks zookeeper is not working on it? From Cloudera Manager all the Hosts and instances are green and shown great.SSH connect to the master, or workers the terminal master, $ shell:HBase shelllistdescribe and all the commands working great, but when trying to do connection and get from outside to master getting the error. checked hbase-site.xml and all the zookeeper nodes are there like internal ip hosts. what else I may check?
Hello, I'm trying to learn NIFI so this is all new to me, I used to work with Talend and I have hard time translating to NIFI. So the main idea: For example is I have two tables in Postgresql Table CITY :ID (auto generated), city_nameTable PERSON :ID (auto generated), first_name, last_name, city_id and I have a CSV file :first_name, last_name, city_name Can you please explain how I can insert in tow tables from one flowfile and refer in the table PERSON to the ID of the city not the name from the table CITY.Thank you.
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.