Member since
02-07-2022
26
Posts
0
Kudos Received
0
Solutions
01-13-2023
07:04 PM
Hi Team, cloudera embedded postgres db taking more memory. How to check and reduce embedded postgres db taking more memory?? Regards, Hanu
... View more
Labels:
01-10-2023
06:03 AM
Hi All, I am also facing this WARN messages in one of my spark job and output not getting generating and job status shows succeeded. any suggestions please???
... View more
12-28-2022
03:51 AM
Hi @AsimShaikh , We are observing on specific nodes
... View more
12-22-2022
10:51 PM
Hi team, How to troubleshoot always particular nodemanagers nodes memory reaches >95% when job is running Regards, Hanu
... View more
- Tags:
- cdh
- node manager
Labels:
- Labels:
-
Cloudera Enterprise Data Hub
12-22-2022
08:44 PM
Hi Team, While running spark job we are getting below error.. any one please guide me on this.. ERROR yarn.ApplicationMaster: User class threw exception: java.lang.IllegalArgumentException: Unable to instantiate SparkSession with Hive support because Hive classes are not found. java.lang.IllegalArgumentException: Unable to instantiate SparkSession with Hive support because Hive classes are not found. Regards, Hanu
... View more
Labels:
- Labels:
-
Cloudera Enterprise Data Hub
12-06-2022
12:26 AM
Hi All, As CDH version was 5.16.1 out of support , i am unable to contact support. please help here.
... View more
12-05-2022
07:11 AM
Hi Team,
Please suggest any guide or recommendations of tuning the memory parameters for spark jobs.
Eg: To process 500GB input data by spark job how much executors and memory required for each executor etc...
... View more
Labels:
12-05-2022
07:04 AM
Hi Team, Spark job is hang or struck due to below error. can any one please help here.. 22/12/05 22:29:55 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=90000 watcher=hconnection-0x5e29988e0x0, quorum=localhost:2181, baseZNode=/hbase 22/12/05 22:29:55 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) 22/12/05 22:29:55 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1126) .... 22/12/05 22:30:12 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) 22/12/05 22:30:12 ERROR zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts 22/12/05 22:30:12 WARN zookeeper.ZKUtil: hconnection-0x5e29988e0x0, quorum=localhost:2181, baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
... View more
Labels:
11-14-2022
08:10 PM
Hi Team, We have 1 table which has more than 1 lakh partitions. Support team moved data from HDFS to other path and try to do MSCK but it is not working even drop table also. Getting below errors.. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask While drooping partitions also getting same error after 10min can one help me on this..
... View more
Labels:
- Labels:
-
Apache Hive
11-14-2022
07:35 PM
@ask_bill_brooks not only for NIFI ,this vulnerability has been raised for below path also.. /opt/hadoop/yarn/nm/filecache /opt/hadoop/yarn/nm/usercache
... View more
11-14-2022
05:42 AM
@Girish007 do you have any update on this CVE-2022-42889 Vulnerability?
... View more
11-14-2022
05:37 AM
Hi Pajoshi, Please find my inline comments. 1) If the replication factor of deleted files was 1. -->Replication factor - 2 2) If there are blocks still pending to be deleted. This could be checked from NN UI ->we observed 6 blocks pending to be deleted. 3) If there are hdfs snapshots configured on the deleted paths or its parent directory --> We deleted hdfs snapshots
... View more
11-14-2022
05:35 AM
not yet issue didn't resolved
... View more
11-04-2022
04:31 AM
Hi Team, We have 3 DN's and Data Directories mount point size is around 1TB on each Node , total Data Directories size was 3TB. I deleted 500GB data from HDFS but space got release from only DN3 Data Directory not from other DN's. Please advise..
... View more
- Tags:
- data
- directories
- HDFS
Labels:
- Labels:
-
HDFS
11-03-2022
12:30 AM
11-03-2022
12:27 AM
Hi Team, Can you please explain me how to interpret 'Cluster CPU' graph from CM.
... View more
Labels:
- Labels:
-
Cloudera Manager
10-21-2022
03:28 AM
Hi Team, We have total 46 nodes cluster. 9(2 Master & 7 slave ) nodes on Baremetal and 37 nodes on VCP(VM's). Can we set any limit/quota on CDH side by restrict using maximum bandwidth from N/W switches??
... View more
Labels:
- Labels:
-
Cloudera Enterprise Data Hub
10-13-2022
05:50 AM
Hey @mszurap while accessing hive from beeline due to below messages it hanged. 2022-10-12 09:41:26,507 INFO org.apache.hadoop.hive.common.JvmPauseMonitor: [org.apache.hadoop.hive.common.JvmPauseMonitor$Monitor@78267545]: Detected pause in JVM or host machine (eg GC): pause of approximately 5107ms GC pool 'PS MarkSweep' had collection(s): count=1 time=3349ms 2022-10-12 09:41:43,321 INFO org.apache.hadoop.hive.common.JvmPauseMonitor: [org.apache.hadoop.hive.common.JvmPauseMonitor$Monitor@78267545]: Detected pause in JVM or host machine (eg GC): pause of approximately 5098ms GC pool 'PS MarkSweep' had collection(s): count=3 time=9940ms 2022-10-12 09:42:20,627 INFO org.apache.hadoop.hive.common.JvmPauseMonitor: [org.apache.hadoop.hive.common.JvmPauseMonitor$Monitor@78267545]: Detected pause in JVM or host machine (eg GC): pause of approximately 6398ms GC pool 'PS MarkSweep' had collection(s): count=6 time=23371ms 2022-10-12 09:42:31,927 INFO org.apache.hadoop.hive.common.JvmPauseMonitor: [org.apache.hadoop.hive.common.JvmPauseMonitor$Monitor@78267545]: Detected pause in JVM or host machine (eg GC): pause of approximately 5057ms GC pool 'PS MarkSweep' had collection(s): count=1 time=3303ms 2022-10-12 09:45:46,227 INFO org.apache.hadoop.hive.common.JvmPauseMonitor: [org.apache.hadoop.hive.common.JvmPauseMonitor$Monitor@78267545]: Detected pause in JVM or host machine (eg GC): pause of approximately 6036ms GC pool 'PS MarkSweep' had collection(s): count=2 time=7653ms 2022-10-12 09:48:53,560 INFO org.apache.hadoop.hive.common.JvmPauseMonitor: [org.apache.hadoop.hive.common.JvmPauseMonitor$Monitor@78267545]: Detected pause in JVM or host machine (eg GC): pause of approximately 5099ms GC pool 'PS MarkSweep' had collection(s): count=40 time=140485ms 2022-10-12 09:54:03,673 INFO org.apache.hadoop.hive.common.JvmPauseMonitor: [org.apache.hadoop.hive.common.JvmPauseMonitor$Monitor@78267545]: Detected pause in JVM or host machine (eg GC): pause of approximately 5401ms GC pool 'PS MarkSweep' had collection(s): count=31 time=111314ms
... View more
10-13-2022
02:49 AM
Hive Metastore server health in node 1 shows un-healthy. because of this jobs are failing. How to trouble shoot this issue??
... View more
10-12-2022
11:10 PM
any one please help me on this
... View more
10-12-2022
11:07 PM
Thank you all who are supporting or helping on this issue
... View more
10-12-2022
11:07 PM
Hi Team, We have 2 Hive Metastore server services configured on cluster in node1 and node2. In node1 Hive Metastore getting alert with below message. "Hive Metastore Canary" The Hive Metastore canary failed to create a database. I am seeing below errors in servicemonitor log file. 2022-10-12 22:40:06,657 WARN com.cloudera.cmf.cdh6client.hive.MetastoreClientImpl: (2 skipped) Could not drop hive database: cloudera_manager_metastore_canary_test_db_hive_HIVEMETASTORE_0137966f79e5f15b3b5d4dec61b7592e com.cloudera.cdh6client.hive.shaded.org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out at com.cloudera.cdh6client.hive.shaded.org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129) at com.cloudera.cdh6client.hive.shaded.org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at com.cloudera.cdh6client.hive.shaded.org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) at com.cloudera.cdh6client.hive.shaded.org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) at com.cloudera.cdh6client.hive.shaded.org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) at com.cloudera.cdh6client.hive.shaded.org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_database(ThriftHiveMetastore.java:770) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_database(ThriftHiveMetastore.java:757) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.dropDatabase(HiveMetaStoreClient.java:940) at com.cloudera.cmf.cdh6client.hive.MetastoreClientImpl.dropDatabase(MetastoreClientImpl.java:163) at com.cloudera.cmon.firehose.polling.hive.HiveMetastoreCanary.cleanUpFromPreviousRuns(HiveMetastoreCanary.java:484) at com.cloudera.cmon.firehose.polling.hive.HiveMetastoreCanary.doWorkWithClientConfig(HiveMetastoreCanary.java:175) at com.cloudera.cmon.firehose.polling.hive.HiveMetastoreCanary.doWorkWithClientConfig(HiveMetastoreCanary.java:52) at com.cloudera.cmon.firehose.polling.AbstractCdhWorkUsingClientConfigs.doWork(AbstractCdhWorkUsingClientConfigs.java:45) at com.cloudera.cmon.firehose.polling.CdhTask$InstrumentedWork.doWork(CdhTask.java:230) at com.cloudera.cmf.cdhclient.util.ImpersonatingTaskWrapper.runTask(ImpersonatingTaskWrapper.java:72) at com.cloudera.cmf.cdhclient.util.ImpersonatingTaskWrapper.access$000(ImpersonatingTaskWrapper.java:21) at com.cloudera.cmf.cdhclient.util.ImpersonatingTaskWrapper$1.run(ImpersonatingTaskWrapper.java:107) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) at com.cloudera.cmf.cdh6client.security.UserGroupInformationImpl.doAs(UserGroupInformationImpl.java:42) at com.cloudera.cmf.cdhclient.util.ImpersonatingTaskWrapper.doWork(ImpersonatingTaskWrapper.java:104) at com.cloudera.cmf.cdhclient.CdhExecutor$1.call(CdhExecutor.java:125) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at com.cloudera.cdh6client.hive.shaded.org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) ... 27 more Please help me on this issue,
... View more
Labels:
- Labels:
-
Apache Hive
10-12-2022
10:23 PM
Hey Team, As suggested by @vaishaakb i done the checks of UUID and properly set the java home then cloudera agent able to communicate with server. Thanks for your help
... View more
09-30-2022
07:21 AM
Hi , We upgraded OS of one of the server in cluster from RHEL 7.5 to 7.9. After that we are unable to start cloudera services from webui, getting below errors in agent log file. can anyone help me to get rid of this issue. AttributeError: 'NoneType' object has no attribute 'get' [30/Sep/2022 19:37:38 +0000] 1048 MainThread agent WARNING Long HB processing time: 7.00362992287 [30/Sep/2022 19:38:01 +0000] 1048 DnsResolutionMonitor throttling_logger INFO DnsTest not running. Java not located. [30/Sep/2022 19:38:37 +0000] 1048 MonitorDaemon-Reporter firehoses INFO Creating a connection to the ACTIVITYMONITOR. [30/Sep/2022 19:38:37 +0000] 1048 MonitorDaemon-Reporter firehoses INFO Creating a connection to the SERVICEMONITOR. [30/Sep/2022 19:38:37 +0000] 1048 MonitorDaemon-Reporter firehoses INFO Creating a connection to the HOSTMONITOR. [30/Sep/2022 19:38:37 +0000] 1048 MonitorDaemon-Reporter throttling_logger ERROR Error sending messages to firehose: mgmt-HOSTMONITOR-d85e01b86fca15e35281bae2797b3c77 Traceback (most recent call last): File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/firehose.py", line 121, in _send self._port) File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/avro/ipc.py", line 469, in __init__ self.conn.connect() File "/usr/lib64/python2.7/httplib.py", line 837, in connect self.timeout, self.source_address) File "/usr/lib64/python2.7/socket.py", line 571, in create_connection raise err error: [Errno 111] Connection refused [30/Sep/2022 19:47:09 +0000] 1048 MainThread heartbeat_tracker INFO HB stats (seconds): num:46 LIFE_MIN:0.00 min:0.00 mean:0.02 max:0.26 LIFE_MAX:0.01 ~
... View more
Labels:
- Labels:
-
Cloudera Manager