Member since
09-25-2018
99
Posts
6
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2029 | 11-03-2021 02:55 AM | |
1614 | 09-21-2020 10:04 PM | |
2865 | 08-14-2020 03:20 AM | |
3927 | 08-20-2019 11:07 PM | |
9542 | 01-06-2019 07:32 PM |
06-18-2020
09:12 PM
Hello All, We have Spark jobs which are run in local mode and are creating a lot of application temporary files which are stored in local /tmp folder, would need some assistance in getting these deleted automatically. Is there some configuration setting in CM that can be used to get this fixed. CDH & CM - 5.16.2
... View more
Labels:
- Labels:
-
Apache Spark
-
Apache YARN
-
Cloudera Manager
04-29-2020
11:35 PM
@GangWar Thanks for your reply, what I am trying to zero in, is why some files (recently put) are getting created with RF 1 rather than RF 3. I have checked multiple sites but failed to get some answers, and hitting the wall. Would appreciate if there are any suggestions / pointer that I could check to fix this issue. My Issue is as below: /User 1/logs/User1Logs >>> files under this folder have replication factor of 1. /User 2/logs/User2Logs >>> files under this folder have replication factor of 1 /User 3/logs/User3Logs >>> files under this folder have replication factor of 3 Regards Wert
... View more
04-29-2020
02:49 AM
Hello, Is there a way to check the replication factor of a particular folder in HDFS? While we have default replication set to 3 in CM for some reason files being uploaded in a particular folder shows up with replication factor of 1. Regards Wert
... View more
Labels:
- Labels:
-
Cloudera Manager
-
HDFS
02-27-2020
06:38 PM
Thanks @venkatsambath & @EricL this would certainly help in my hadoop journey. Regards
... View more
02-26-2020
08:37 PM
Hello, I would like to understand the internal in / outs of Cloudera flow. How CM interacts with other components in the Hadoop stack ( HDFS / YARN/ IMPALA etc.), would like to know if there are any references material / documents which I can go through to enhance my knowledge in this area. Appreciate any information in this regard. Regards
... View more
Labels:
11-20-2019
07:51 PM
@bgooley Thanks for your reply and assistance. @EricL Current heap from RM is 1GB Regards Wert
... View more
11-19-2019
11:40 PM
Hello All, Off late we have been getting concerning alerts for Reports Manager (The health test result for REPORTS_MANAGER_PAUSE_DURATION has become concerning: Average time spent paused was 35.6 second(s) (59.41%) per minute over the previous 5 minute(s). Warning threshold: 30.00%.) occasionally reports manager service also goes to bad state. I believe, this has to do with Reports Manager Heap Memory, but am in a doubt hence wanted to get a confirmation before playing around with it. I have attached a snippet of the error, which I get. Also would like to know if there is any reference guide / document by which, we can calculate and periodically adjust such items. HDFS Used – 10TB / Total 90TB FSImage – 980M CM - 6.16.2 / CDH - 5.16.2 The health test result for REPORTS_MANAGER_PAUSE_DURATION has become concerning: Average time spent paused was 35.6 second(s) (59.41%) per minute over the previous 5 minute(s). Warning threshold: 30.00%.
[root@MY Server cloudera-scm-headlamp]# tail -n 300 mgmt-cmf-mgmt-REPORTSMANAGER-MY Server.com.log.out
2019-11-20 19:03:43,443 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1363ms: GC pool 'ParNew' had collection(s): count=1 time=65ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1537ms
2019-11-20 19:03:48,933 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1965ms: GC pool 'ParNew' had collection(s): count=1 time=834ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1571ms
2019-11-20 19:04:10,282 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1228ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1280ms
2019-11-20 19:04:15,629 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1345ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1366ms
2019-11-20 19:04:26,295 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1321ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1343ms
2019-11-20 19:04:31,499 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1114ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1252ms
2019-11-20 19:04:36,891 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1390ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1399ms
2019-11-20 19:04:52,016 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1138ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1158ms
2019-11-20 19:05:12,572 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1014ms: GC pool 'ParNew' had collection(s): count=1 time=1ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1226ms
2019-11-20 19:05:28,066 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1051ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1311ms
2019-11-20 19:05:52,991 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1325ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1332ms
2019-11-20 19:06:35,411 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1104ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1395ms
2019-11-20 19:06:46,384 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1083ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1421ms
2019-11-20 19:06:51,484 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1098ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1252ms
2019-11-20 19:07:07,179 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1249ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1273ms
2019-11-20 19:07:27,892 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1103ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1258ms
2019-11-20 19:07:33,073 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1179ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1294ms
2019-11-20 19:07:38,111 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1036ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1096ms
2019-11-20 19:07:44,198 INFO com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
2019-11-20 19:07:45,318 INFO com.cloudera.cmf.BasicScmProxy: Authentication to SCM required.
2019-11-20 19:07:45,394 INFO com.cloudera.cmf.BasicScmProxy: Using encrypted credentials for SCM
2019-11-20 19:07:45,403 INFO com.cloudera.cmf.BasicScmProxy: Authenticated to SCM.
2019-11-20 19:08:07,837 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1157ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1218ms
2019-11-20 19:08:28,276 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1133ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1248ms
2019-11-20 19:08:33,105 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1227ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1251ms
2019-11-20 19:08:38,015 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1179ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1336ms
2019-11-20 19:08:40,247 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1175ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1312ms
2019-11-20 19:08:44,598 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1156ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1284ms
2019-11-20 19:08:46,743 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1094ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1312ms
2019-11-20 19:08:53,086 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1162ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1340ms
2019-11-20 19:08:55,207 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1120ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1306ms
2019-11-20 19:08:57,373 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1109ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1241ms
2019-11-20 19:08:59,443 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1069ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1203ms
2019-11-20 19:09:03,683 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1171ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1307ms
2019-11-20 19:09:05,929 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1191ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1355ms
2019-11-20 19:09:12,724 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1046ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1235ms
2019-11-20 19:09:14,848 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1123ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1292ms
2019-11-20 19:09:17,095 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1247ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1373ms
2019-11-20 19:09:20,294 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1142ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1283ms
2019-11-20 19:09:27,791 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1084ms: GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1524ms
2019-11-20 19:09:30,897 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1050ms: GC pool 'ParNew' had collection(s): count=1 time=1ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1358ms
2019-11-20 19:09:48,006 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1039ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1283ms
2019-11-20 19:09:50,625 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1118ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1155ms
2019-11-20 19:10:05,535 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1039ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1174ms
2019-11-20 19:10:12,452 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1164ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1268ms
2019-11-20 19:10:24,685 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1039ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1234ms
2019-11-20 19:10:34,427 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1163ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1312ms
2019-11-20 19:10:39,574 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1083ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1419ms
2019-11-20 19:10:49,650 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1198ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1413ms
2019-11-20 19:10:54,658 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1006ms: GC pool 'ParNew' had collection(s): count=1 time=1ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1184ms
2019-11-20 19:10:59,817 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1081ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1249ms
2019-11-20 19:11:10,429 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1215ms: GC pool 'ParNew' had collection(s): count=1 time=1ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1294ms
2019-11-20 19:11:20,762 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1194ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1237ms
2019-11-20 19:11:25,839 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1075ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1331ms
2019-11-20 19:11:31,441 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1057ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1378ms
2019-11-20 19:11:36,533 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1065ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1200ms
2019-11-20 19:11:51,468 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1064ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1352ms
2019-11-20 19:11:57,174 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1189ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1529ms
2019-11-20 19:12:02,353 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1162ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=2 time=1340ms
2019-11-20 19:12:07,629 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1273ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1436ms
2019-11-20 19:12:18,074 INFO com.cloudera.enterprise.debug.JvmPauseMonitor: Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 1127ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=1252ms
2019-11-20 19:12:27,859 INFO com.cloudera.headlamp.LuceneImageVisitor: Wrote '/var/lib/cloudera-scm-headlamp/hdfs/nameservice1/aggregates.tmp' (25361 bytes)
2019-11-20 19:12:27,859 INFO com.cloudera.headlamp.LuceneImageVisitor: Starting publishing HBase metrics
2019-11-20 19:12:27,859 INFO com.cloudera.headlamp.LuceneImageVisitor: Finished publishing HBase metrics
2019-11-20 19:12:27,859 INFO com.cloudera.headlamp.HeadlampIndex: Temporarily shutting down search service
2019-11-20 19:12:27,859 INFO com.cloudera.headlamp.HeadlampServiceImpl: Saving existing quotas for later merge
2019-11-20 19:12:27,859 INFO com.cloudera.headlamp.HeadlampServiceImpl: Getting all files with quotas of type ALL
2019-11-20 19:12:27,859 INFO com.cloudera.headlamp.HeadlampServiceImpl: Performing query: +(DSQUOTA:[0 TO *} NSQUOTA:[0 TO *}) sortBy: null sortRerverse: false limit: 10000
2019-11-20 19:12:27,861 INFO com.cloudera.headlamp.HeadlampServiceImpl: Got 1 results for +(DSQUOTA:[0 TO *} NSQUOTA:[0 TO *})
2019-11-20 19:12:27,861 INFO com.cloudera.headlamp.HeadlampServiceImpl: Search for '+(DSQUOTA:[0 TO *} NSQUOTA:[0 TO *})' took 1ms
2019-11-20 19:12:27,862 INFO com.cloudera.headlamp.HeadlampIndex: Swapping index and FS image files
2019-11-20 19:12:27,862 INFO com.cloudera.headlamp.HeadlampIndex: Deleting old FS Image (/var/lib/cloudera-scm-headlamp/hdfs/nameservice1/fsimage)
2019-11-20 19:12:28,121 INFO com.cloudera.headlamp.HeadlampIndex: Moving new FS Image in place.
2019-11-20 19:12:28,121 INFO com.cloudera.headlamp.HeadlampIndex: Deleting old Index (/var/lib/cloudera-scm-headlamp/hdfs/nameservice1/index)
2019-11-20 19:12:30,733 INFO com.cloudera.headlamp.HeadlampIndex: Moving new Index in place.
2019-11-20 19:12:30,733 INFO com.cloudera.headlamp.HeadlampIndex: Deleting old Aggregate Summary (/var/lib/cloudera-scm-headlamp/hdfs/nameservice1/aggregates)
2019-11-20 19:12:30,734 INFO com.cloudera.headlamp.HeadlampIndex: Moving new Aggregate Summary in place.
2019-11-20 19:12:30,734 INFO com.cloudera.headlamp.HeadlampIndex: Starting up search service
2019-11-20 19:12:30,803 INFO com.cloudera.headlamp.HeadlampServiceImpl: Performing query: PATH:/ sortBy: null sortRerverse: false limit: 10000
2019-11-20 19:12:30,804 INFO com.cloudera.headlamp.HeadlampServiceImpl: Got 1 results for PATH:/
2019-11-20 19:12:30,804 INFO com.cloudera.headlamp.HeadlampServiceImpl: Search for 'PATH:/' took 1ms
2019-11-20 19:12:30,804 INFO com.cloudera.headlamp.HeadlampServiceImpl: Updating DSQUOTA in quota cache for '/' from -1 to -1
2019-11-20 19:12:30,980 INFO com.cloudera.headlamp.HeadlampServiceImpl: Performing query: PATH:/ sortBy: null sortRerverse: false limit: 10000
2019-11-20 19:12:30,981 INFO com.cloudera.headlamp.HeadlampServiceImpl: Got 1 results for PATH:/
2019-11-20 19:12:30,981 INFO com.cloudera.headlamp.HeadlampServiceImpl: Search for 'PATH:/' took 1ms
2019-11-20 19:12:30,981 INFO com.cloudera.headlamp.HeadlampServiceImpl: Updating NSQUOTA in quota cache for '/' from 9223372036854775807 to 9223372036854775807
2019-11-20 19:12:32,272 INFO com.cloudera.headlamp.HeadlampIndexManager: Finished indexing HDFS services
2019-11-20 19:38:45,654 INFO com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
2019-11-20 19:38:46,655 INFO com.cloudera.cmf.BasicScmProxy: Authentication to SCM required.
2019-11-20 19:38:46,755 INFO com.cloudera.cmf.BasicScmProxy: Using encrypted credentials for SCM
2019-11-20 19:38:46,874 INFO com.cloudera.cmf.BasicScmProxy: Authenticated to SCM. Appreciate any help in this regard Regards Wert
... View more
Labels:
- Labels:
-
Cloudera Manager
11-05-2019
08:14 PM
Hi @gsthina I really appreciate all your time & assistance this topic. Thanks a lot. Regards Wert
... View more
11-05-2019
05:15 PM
Hi Gst, Appreciate your assistance on this so far, regarding stopping Yarn Service in our cluster it would be difficult as we have Spark streaming jobs running, stopping those would require a lot of approvals etc, any other way to get the contents inside /data/yarn/nm/usercache/my.application/filecache/* deleted. Regards Wert
... View more