HDP 2.3.6 and HDP 2.4.2 are pretty much the same except for Hive 1.2.1 and Spark 1.6. Kafka mirror maker is now supported and perhaps your issue is resolved in 2.3.6 as well. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.6/bk_HDP_RelNotes/content/new_features.html I think however there's a documentation issue in the release notes that contradicts this statement and I will follow up. Please try upgrading to 2.3.6 or test this on another cluster with 2.3.6 to confirm. ... View more

@Nic Hopper yes so now you're good to go as you have access to HDFS via CLI (Command Shell). You can now upload and download files using CLI sudo -u username hdfs dfs -put file /user/username/ sudo -u username hdfs dfs -get file /user/username/file if you have your username available locally on the OS then you can skip the part with sudo -u username just change to your userID and run those commands su username hdfs dfs -ls hdfs dfs -put hdfs dfs -get also as a good practice, please accept the answer to close this thread. ... View more

Run the following command, then repeat the earlier commands sudo -u hdfs hdfs dfsadmin -safemode leave ... View more

Please consider publishing an article on this, others will find it useful as it's not an obvious find. ... View more

@Biswajit Chakraborty I can't believe I didn't think of this earlier, take a look at https://apache.googlesource.com/mrunit/+/e43ef01dd1199a7eb0963edbf05258a8609bf0dc/src/test/java/org/apache/hadoop/mrunit/mapreduce/TestDistributedCache.java This is MRUnit's own DistributedCache test with examples how to set it up. ... View more

unfortunately there are none that I know of. We do have one engineer who's providing support for his unit-testing framework and it has mapreduce unit testing capabilities, though I am not sure if distributedcache testing is supported. https://github.com/sakserv/hadoop-mini-clusters ... View more