this is what we have in hadoop-end from amabri GUI in HDFS service hadoop-env-hdfs.txt ... View more

this is what we have in kafka-end ( in ambari GUI ) #!/bin/bash # Set KAFKA specific environment variables here. # The java implementation to use. export JAVA_HOME={{java64_home}} export PATH=$PATH:$JAVA_HOME/bin export PID_DIR={{kafka_pid_dir}} export LOG_DIR={{kafka_log_dir}} export KAFKA_KERBEROS_PARAMS={{kafka_kerberos_params}} export JMX_PORT=9997 # Add kafka sink to classpath and related depenencies if [ -e "/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar" ]; then export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar export CLASSPATH=$CLASSPATH:/usr/lib/ambari-metrics-kafka-sink/lib/* fi if [ -f /etc/kafka/conf/kafka-ranger-env.sh ]; then . /etc/kafka/conf/kafka-ranger-env.sh fi ... View more

@amarnath reddy pappu the problem is that I not sure what is JVM parameters variable for HDFS and kafka - which Variable I need to search ? if not defined should I add this variable ? ... View more

@Jay , actually I was asking about disk configuration as RAID configuration , lets say we have cluster HDP 2.6.4 with that details: 1) Worker-> (OS) Raid1 + Raid 5 2) Master -> (OS) Raid1 + Raid10 3) Kafka-> (OS) Raid1 + Raid10 based on that connfiguration , can we upgrade the HDP to 3.0 version ? dose HDP 3.0 support the above configuration ... View more

@jay regarding to upgrade to 3.0 version , we have versions as 2.6.4 so I guess it can be upgrade to 3.0 , do we need also to make storage changes in machines in the clusters ? , or just to upgrade the HDP without any disks changes ? ... View more

@Jay , thank you so much we do the testing and they are fine , now we need to do the steps VIA API ... View more

@Jay , great let me check your procedure on my cluster , I will update soon ... ... View more

@Akhil any suggestion regarding my comments ? ... View more

but files still not effected , seems this conf isnt work , or I miss something? ls -ltr total 1592376 -rw-r--r--. 1 mapred hadoop 268437275 Aug 22 2017 mapred-mapred-historyserver-master.sys64.com.log.6 -rw-r--r--. 1 mapred hadoop 268435538 Aug 28 2017 mapred-mapred-historyserver-master.sys64.com.log.5 -rw-r--r--. 1 mapred hadoop 268435639 Sep 2 2017 mapred-mapred-historyserver-master.sys64.com.log.4 -rw-r--r--. 1 mapred hadoop 268435470 May 30 10:02 mapred-mapred-historyserver-master.sys64.com.log.3 -rw-r--r-- 1 mapred hadoop 268441217 Jul 10 02:45 mapred-mapred-historyserver-master.sys64.com.log.2 -rw-r--r-- 1 mapred hadoop 268442505 Jul 16 01:17 mapred-mapred-historyserver-master.sys64.com.log.1 -rw-r--r-- 1 mapred hadoop 1477 Jul 19 15:28 mapred-mapred-historyserver-master.sys64.com.out.5 -rw-r--r-- 1 mapred hadoop 1687 Jul 25 17:51 mapred-mapred-historyserver-master.sys64.com.out.4 -rw-r--r-- 1 mapred hadoop 1477 Jul 29 15:17 mapred-mapred-historyserver-master.sys64.com.out.3 -rw-r--r-- 1 mapred hadoop 1477 Aug 1 19:30 mapred-mapred-historyserver-master.sys64.com.out.2 -rw-r--r-- 1 mapred hadoop 1687 Aug 7 11:48 mapred-mapred-historyserver-master.sys64.com.out.1 -rw-r--r-- 1 mapred hadoop 1477 Aug 8 17:09 mapred-mapred-historyserver-master.sys64.com.out -rw-r--r-- 1 mapred hadoop 19888308 Aug 8 17:20 mapred-mapred-historyserver-master.sys64.com.log ... View more