@Jason4Ever : Please if your server is able to connect with internet by doing some PING commands ... View more

@PentaReddy: I guess table is not getting dropped. Please run repair statement and the run select statement to retrive the data.    MSCK [REPAIR] TABLE table_name [ADD/DROP/SYNC PARTITIONS];   ... View more

@Bharath Kumar: Yes, you can create no-login them in AD. Technically, they should be login accounts if you are planning to run some service. That may vary based on the senario ... View more

@Bharath N Try to perform the following steps on the failed DataNode: Get the list of DataNode directories from /etc/hadoop/conf/hdfs-site.xml using the following command: $ grep -A1 dfs.datanode.data.dir /etc/hadoop/conf/hdfs-site.xml <name>dfs.datanode.data.dir</name> <value>/data0/hadoop/hdfs/data,/data1/hadoop/hdfs/data,/data2/hadoop/hdfs/data, /data3/hadoop/hdfs/data,/data4/hadoop/hdfs/data,/data5/hadoop/hdfs/data,/data6/hadoop/hdfs/data, /data7/hadoop/hdfs/data,/data8/hadoop/hdfs/data,/data9/hadoop/hdfs/data</value> Get datanodeUuid by grepping the DataNode log: $ grep "datanodeUuid=" /var/log/hadoop/hdfs/hadoop-hdfs-datanode-$(hostname).log | head -n 1 | perl -ne '/datanodeUuid=(.*?),/ && print "$1\n"' 1dacef53-aee2-4906-a9ca-4a6629f21347 Copy over a VERSION file from one of the <dfs.datanode.data.dir>/current/ directories of a healthy running DataNode: $ scp <healthy datanode host>:<dfs.datanode.data.dir>/current/VERSION ./ Modify the datanodeUuid in the VERSION file with the datanodeUuid from the above grep search: $ sed -i.bak -E 's|(datanodeUuid)=(.*$)|\1=1dacef53-aee2-4906-a9ca-4a6629f21347|' VERSION Blank out the storageID= property in the VERSION file: $ sed -i.bak -E 's|(storageID)=(.*$)|\1=|' VERSION Copy this modified VERSION file to the current/ path of every directory listed in dfs.datanode.data.dir property of hdfs-site.xml: $ for i in {0..9}; do cp VERSION /data$i/hadoop/hdfs/data/current/; done Change permissions on this VERSION file to be owned by hdfs:hdfs with permissions 644: $ for i in {0..9}; do chown hdfs:hdfs /data$i/hadoop/hdfs/data/current/VERSION; done $ for i in {0..9}; do chmod 664 /data$i/hadoop/hdfs/data/current/VERSION; done One more level down, there is a different VERSION file located under the Block Pool current folder at: /data0/hadoop/hdfs/data/current/BP-*/current/VERSION This file does not need to be modified -- just place then in the appropriate directories. Copy over this particular VERSION file from a healthy DataNode into the current/BP-*/current/ folder for each directory listed in dfs.datanode.data.dir of hdfs-site.xml: $ scp <healthy datanode host>:<dfs.datanode.data.dir>/current/BP-*/current/VERSION ./VERSION2 $ for i in {0..9}; do cp VERSION2 /data$i/hadoop/hdfs/data/current/BP-*/current/VERSION; done Change permissions on this VERSION file to be owned by hdfs:hdfs with permissions 644: $ for i in {0..9}; do chown hdfs:hdfs /data$i/hadoop/hdfs/data/current/BP-*/current/VERSION; done $ for i in {0..9}; do chmod 664 /data$i/hadoop/hdfs/data/current/BP-*/current/VERSION; done Restart DataNode from Ambari. The VERSION file located at <dfs.datanode.data.dir>/current/VERSION will have its storageID repopulated with a regenerated ID. If any data is not an issue (say, for example, the node was previously in a different cluster, or was out of service for an extended time), then delete all data and directories in the dfs.datanode.data.dir (keep that directory, though), restart the data node daemon or servic ... View more

@SH Kim Did you try to do a graceful shutdown of region servers and Datanodes decommissioning? as you are using very less number of nodes, always it better to have more than 50% availability. ... View more

@vishal dutt Spark driver is not able to find the sqljdbc.jar in class path. When using spark-submit , the application jar along with any jars included with the --jars option will be automatically transferred to the cluster. URLs supplied after --jars must be separated by commas. That list is included in the driver and executor classpaths. Directory expansion does not work with --jars . Else 1) Provide the spark.driver.extraClassPath =/usr/hdp/hive/lib/mysql-connector-java.jar 2) Provide the spark.executor.extraClassPath = /usr/hdp/hive/lib/mysql-connector-java.jar. 3) Add Sqljdbc.jar to the Spark Classpath or add it using -jar option. Hope this helps you. ... View more

@David Manukian Try to see if process if running or not ps -ef | grep ambari-server see whether port is lessoning lsof -nalp | grep 8080 Then if everything is working as expected, the try with full qualified domain name like FQDN:8080 Hope this helps you. ... View more