Member since
08-08-2017
1652
Posts
30
Kudos Received
11
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2093 | 06-15-2020 05:23 AM | |
| 17459 | 01-30-2020 08:04 PM | |
| 2256 | 07-07-2019 09:06 PM | |
| 8728 | 01-27-2018 10:17 PM | |
| 4914 | 12-31-2017 10:12 PM |
07-10-2018
11:22 AM
hi all we have issue on data-node machine ( worker machine ) we notice that /var became full and that because stdout file is
89G example: /var/Hadoop/yarn/log/application
183625335_0110/container_e54_1532846235_0180_02_025216/stdout please advise what could be the reason for this issue and
how to solve it so stdout will not became as huge size
... View more
Labels:
07-09-2018
08:59 PM
@Jay another question please , dose my ambari version as I mentioned up , are ok for HDP upgrade to 2.6.5 ?
... View more
07-09-2018
06:55 AM
@Jay chould you please help us with the follwing thred ? - https://community.hortonworks.com/questions/202037/hive-many-hive-process-was-open-on-namenode-machin.html
... View more
07-09-2018
06:31 AM
we have ambari cluster with HDP version 2.6.4 we want to upgrade HDP to 2.6.5 what are the prerequisite for this upgrade ? ambari current version: rpm -qa | grep ambari
ambari-agent-2.6.1.0-143.x86_64
ambari-metrics-grafana-2.6.1.0-143.x86_64
ambari-metrics-collector-2.6.1.0-143.x86_64
ambari-metrics-monitor-2.6.1.0-143.x86_64
ambari-metrics-hadoop-sink-2.6.1.0-143.x86_64
ambari-server-2.6.1.0-143.x86_64
... View more
Labels:
07-09-2018
06:18 AM
not have the jstack command on my linux machine , should I download this cli from redhat repo?
... View more
07-08-2018
08:09 PM
we have ambari cluster version 2.6.0.x we noitce that free memory on master01 machine is 0 , and that because many of the following process ( from ps -ef | grep java ) ambari-+ 65369 65322 0 Jul06 ? 00:01:52 /usr/jdk64/jdk1.8.0_112/bin/java -Xmx1024m -Dhdp.version=2.6.0.3-8 -Djava.net.preferIPv4Stack=true -Dhdvar/log/hadoop/ambari-qa -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.6.0.3-8/hadoop -Dhadoop.id.str=ambari-qa -Dhadoop.root.logger=INFO,c.6.0.3-8/hadoop/lib/native/Linux-amd64-64:/usr/hdp/2.6.0.3-8/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -.config.file=/usr/hdp/2.6.0.3-8/hive/conf/parquet-logging.properties -Dlog4j.configuration=beeline-log4j.properties -Dhadoop.security.logger=INFO,NullA/usr/hdp/2.6.0.3-8/hive/lib/hive-beeline-1.2.1000.2.6.0.3-8.jar org.apache.hive.beeline.BeeLine -u jdbc:hive2://master01.sys748.com:10000/;transportMo any idea why all this process ( around 350 process was opened ? ) , and take all memory on master01 machine [root@master01 ~]# ps -ef | grep java | wc -l
359
other way to show the process :
ps -ef | sed 's/-D[^ ]*//g;s/-X[^ ]*//g;s#^.*/bin/java##g;s/[^ ]*.jar//g;s/^[ ]*//g' |more
ambari-+ 50648 50646 0 Jul06 ? 00:00:00 -bash -c export PATH='/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/
*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/bin/:/usr/bin/:/usr/lib/hive/bin
/:/usr/sbin/' ; ! beeline -u 'jdbc:hive2://master01.sys748.com:10000/;transportMode=binary' -e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused'
-e 'Invalid URL'
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hadoop
-
Apache Hive
07-04-2018
09:11 AM
@Jay I not have the variables in YARN - yarn.nodemanager.localizer.cache.target-size-mb , yarn.nodemanager.localizer.cache.cleanup.interval-ms: so please advice how to add them ? and what is the values that I need to set each of them ? note - /var is 100G on each worker machine
... View more
07-04-2018
09:01 AM
hi all we have hadoop cluster version - 2.6.0.3 with yarn version - 2.7.3 we see that /var in workers ( data node ) machine is full and the root cause for this is that we see huge folders - lockmgr-b1ed0e9c-5700-4575-aa5e-182146f743d9 under /var/hadoop/yarn/local/usercache/hdfs/appcache/application_1530106922052_0041 please advice how to avoid this isshu , why folder are in that huge capasity ? [root@worker01 application_1530106922052_0041]# pwd
/var/hadoop/yarn/local/usercache/hdfs/appcache/application_1530106922052_0041
[root@worker01 application_1530106922052_0041]# \ls -ltr
total 8
drwx--x--- 2 yarn hadoop 6 Jul 3 06:19 filecache
drwxr-xr-x 66 yarn hadoop 4096 Jul 3 06:24 blockmgr-b1ed0e9c-5700-4575-aa5e-182146f743d9
drwxr-xr-x 65 yarn hadoop 4096 Jul 4 08:02 blockmgr-c6530cea-1e98-419b-8653-3e9b467ac029
[root@worker01 application_1530106922052_0041]# du -sh *
33G blockmgr-b1ed0e9c-5700-4575-aa5e-182146f743d9
31G blockmgr-c6530cea-1e98-419b-8653-3e9b467ac029
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Apache YARN
06-28-2018
10:18 AM
finally we success to decrease it by restart the postgresql and set the right values in postgresql.conf
... View more
06-27-2018
09:58 AM
Hi Geoffrey , please advice which lines in postgresql.conf we need to comment ? ( archiving/replication )
... View more