@Rajesh Wonderful article. Just added code block for the commands. ... View more

@uri ben-ari Please try something like following: # curl -sH "X-Requested-By: ambari" -u admin:admin -i http://localhost:8080/api/v1/hosts?fields=Hosts/host_name,Hosts/ip | grep -A 1 host_name | awk '{print $NF}' > /tmp/hosts_details.txt; sed -e '1,2d' -e s'/--//g' -e 's/\n//g' -e 's/"//g' -e '/^$/d' /tmp/hosts_details.txt | awk 'NR%2{printf "%s ",$0;next;}1 . Example Output: # curl -sH "X-Requested-By: ambari" -u admin:admin -i http://amb25101.example.com:8080/api/v1/hosts?fields=Hosts/host_name,Hosts/ip | grep -A 1 host_name | awk '{print $NF}' > /tmp/hosts_details.txt; sed -e '1,2d' -e s'/--//g' -e 's/\n//g' -e 's/"//g' -e '/^$/d' /tmp/hosts_details.txt | awk 'NR%2{printf "%s ",$0;next;}1' amb25101.example.com 172.10.116.149 amb25102.example.com 172.10.116.148 amb25103.example.com 172.10.116.151 amb25104.example.com 172.10.116.150 . ... View more

@Kasim Shaik The error is : WARN namenode.NameNode: Encountered exception during format: org.apache.hadoop.hdfs.qjournal.client.QuorumException: Could not format one or more JournalNodes. 1 exceptions thrown: 10.104.10.16:8485: Cannot create directory /home/kasim/dfs/jn/ha-cluster/current . - Please check the permission on the directory, The user who is running the NameNode format should be able to write to that directory. # ls -ld /home/kasim/dfs/ # ls -ld /home/kasim/dfs/jn # ls -ld /home/kasim/dfs/jn/ha-cluster # ls -ld /home/kasim/dfs/jn/ha-cluster/current # ls -lart /home/kasim/dfs/jn/ha-cluster/current . ... View more

@Sheetal Sharma LongWritable is the WritableComparable for longs, Similarly IntWritable is a WritableComparable for ints. These interfaces [1] & [2] are all necessary for Hadoop/MapReduce, as the Comparable interface is used for comparing when the reducer sorts the keys, and Writable can write the result to the local disk. It does not use the java Serializable because java Serializable is too big or too heavy for hadoop, Writable can serializable the hadoop Object in a very light way. [1] https://hadoop.apache.org/docs/r2.7.2/api/org/apache/hadoop/io/LongWritable.html [2] https://hadoop.apache.org/docs/r2.7.2/api/org/apache/hadoop/io/IntWritable.html#IntWritable() "Comparable" is the interface whose abstract methods give us the flexibility to compare two objects. "Writable" is meant for writing the data to local disk and it's a serialization format. One can implement own Writables in Hadoop. Java’s serialization is too bulky and slow on the system. That’s why Hadoop community had put Writable in place. "WritableComparable" is a combination of the above two interfaces. "int" is a primitive type so it cannot be used as key-value. Integer is the wrapper class around it. So I’ll correct your question that what is the difference between Integer and IntWritable? "IntWritable" is the Hadoop variant of Integer which has been optimized for serialization in the Hadoop environment. An integer would use the default Java Serialization which is very costly in Hadoop environment. . ... View more