@rajdip chaudhuri @Rushikesh Deshmukh I have accepted this answer as it has lot of good information ... View more

@Prakash Punj Did you copy the file locally instead hdfs as I mentioned in my reply? ... View more

@Cecilia Posadas Please read https://www.slideshare.net/mobile/martyhall/hadoop-tutorial-oozie ... View more

Avro 1.8.2 is now available ... View more

Nope, reducers don't communicate with each other and neither the mappers do. All of them runs in a separate JVM containers and don't have information of each other. AppMaster is the demon which takes care and manage these JVM based containers (Mapper/Reducer). ... View more

Hi, I'd like to share a situation we encountered where 99% of our HDFS blocks were reported missing and we were able to recover them. We had a system with 2 namenodes with high availability enabled. For some reason, under the data folders of the datanodes, i.e /data0x/hadoop/hdfs/data/current - we had 2 Block Pools folders listed (example of such folder is BP-1722964902-1.10.237.104-1541520732855). There was one folder containing the IP of namenode1 and another containing the IP of namenode 2. All the data was under the BlockPool of namenode 1, but inside the VERSION files of the namenodes (/data0x/hadoop/hdfs/namenode/current/) the BlockPool id and the namespace ID were of namenode 2 - the namenode was looking for blocks in the wrong block pool folder. I don't know how we got to the point of having 2 block pools folders, but we did. In order to fix the problem - and get HDFS healthy again - we just needed to update the VERSION file on all the namenode disks (on both NN machines) and on all the journal node disks (on all JN machines), to point to Namenode 1. We then restarted HDFS and made sure all the blocks are reported and there's no more missing blocks. ... View more

@Avery Long https://dzone.com/articles/hive-data-types actually a good description for string data types including length here http://hadooptutorial.info/hive-data-types-examples/#String_Data_Types ... View more

I run into the same problem, where the Ambari says it's installed, but the sqoop directory is not there on the data nodes. I am running in a cluster, but it should be the same for sandbox. The current answer does not address this, but the only way to fix this is to uninstall the sqoop client, and re-install it with Ambari. Unfortunately, current web UI does not allow uninstall of clients. Fortunately, you can do it through API calls. Command Syntax is follows: URL=https://${AMBARI_HOST}/api/v1/clusters/${CLUSTER_NAME}/hosts/${HOST_FQDN}/host_components/SQOOP curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X DELETE $URL After that, you can re-install the sqoop client from the Web UI. ... View more

Multicluster mode in Ambari is perhaps one of the most requested features. However its a BIG implementation effort. ... View more

@Pradeep kumar no worries, he has faster wifi connection ... View more