Member since
04-30-2020
15
Posts
1
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2282 | 09-22-2020 05:44 AM | |
4011 | 09-17-2020 05:33 AM |
12-07-2020
05:53 AM
Hi. I have cluster with 12 nodes(9 of them with datanodes). On each host about 30tb capacity, used on datanodes ~15TB (20Vcores on each and 280gb ram). After restart one of VM with DN, datanode was started about 1.5 hours. Is it normal? Or how can i decrease this time.
... View more
Labels:
- Labels:
-
HDFS
12-07-2020
05:46 AM
Hi. I have a problem with outOfMemory error on impala catalog server. Data in tables about 30TB and about 175k partitiions. I set Java Heap Size of Catalog Server in Bytes=30gb, but still have out_of_memory error in start. How can i calculate how much memory needs for catalog server? And is it have any limit to set ?
... View more
Labels:
- Labels:
-
Apache Impala
11-17-2020
07:11 AM
Hello. I configurate TLS certs for HUE(from CA), on hdfs i used AUTO_TLS certs. My config Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini [desktop] app_blacklist= ssl_validate=false use_new_editor=true [notebook] [[interpreters]] [[ldap]] sync_groups_on_login=true show_notebooks=true [spark] livy_server_url=http://myhost:8998 interface=livy [pyspark] name=PySpark interface=livy [r] name=R interface=livy [hadoop] [[hdfs_clusters]] [[[default]]] ssl_cert_ca_verify = false [[ssl]] validate=false [impala] [[ssl]] validate=false In hue i see this:
... View more
Labels:
- Labels:
-
Cloudera Hue
-
HDFS
10-23-2020
02:35 AM
Yes, i thought about cron. But maybe smth in yarn conf can help, but i miss it. Maximum log files bk doesnt help
... View more
10-16-2020
07:51 AM
Hello. I have cluster 9 nodes (400gb memory on each, 2 masters with NN). Capacity will be increase every time(i add ~10TB every week). For 31millon files used ~9gb of 15. What maximum java heap size for NN i can set and what recommended blocksize for big cluster(now its 128mb, should i set 256/512 ?)
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Cloudera Manager
10-13-2020
07:23 AM
Hello. In my cloudera cluser i'm having constantly working script(hes never stop) and logs in local dir's always grows ( local folder /yarn/nm/). [root@sdl-masternode ~]# du -s -h /yarn/nm/usercache/azat/appcache/*
658M /yarn/nm/usercache/azat/appcache/application_1599754680433_0192
2.1G /yarn/nm/usercache/azat/appcache/application_1599754680433_0203
12K /yarn/nm/usercache/azat/appcache/application_1599754680433_0210
6.7G /yarn/nm/usercache/azat/appcache/application_1599754680433_0241
605M /yarn/nm/usercache/azat/appcache/application_1599754680433_0254
40K /yarn/nm/usercache/azat/appcache/application_1600498608856_0120
2.3G /yarn/nm/usercache/azat/appcache/application_1600498608856_0150
12G /yarn/nm/usercache/azat/appcache/application_1600498608856_0155
1.2G /yarn/nm/usercache/azat/appcache/application_1600498608856_0381
4.6G /yarn/nm/usercache/azat/appcache/application_1600498608856_0386
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0398
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0400
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0613
46M /yarn/nm/usercache/azat/appcache/application_1600498608856_0688
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0966
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0974
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0983
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0986
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0989
12K /yarn/nm/usercache/azat/appcache/application_1600498608856_0991
16K /yarn/nm/usercache/azat/appcache/application_1600498608856_1002
96K /yarn/nm/usercache/azat/appcache/application_1600498608856_1003 How can i automatically clean it, for example, if log older then 7 days - its must be retain from local.
... View more
Labels:
- Labels:
-
Apache YARN
-
Cloudera Manager
09-22-2020
07:16 AM
Hello. I have activity monitor in green state. In logs I have smth like this: ps Sorry for quality, have no internet access in my cluster. But when I open navigator web ui I see this:
... View more
Labels:
- Labels:
-
Cloudera Navigator
09-19-2020
05:51 AM
Hello. I need advice there I can see log(audit) of hdfs operations. For example(mkdir, putFromLocal, delete file,change file). I tried use command from cmd "hdfs dfs -mkdir /tmp/mydir" - and don't see this log. There can I find it?
... View more
Labels:
- Labels:
-
HDFS
09-17-2020
05:33 AM
1 Kudo
Miracle! It's start work while I delete file "config.ini.orig" from folder /etc/cloudera-scm/agent/ Now there only "config.ini" file.
... View more