sudo -u hdfs hdfs dfs -mkdir /user/root sudo -u hdfs hdfs dfs -chown -R root:hdfs /user/root ... View more

@Guilherme Braccialli can you add more detail to the answer, used this thread as reference and it's unclear what solution is. ... View more

You can list the app in yarn CLI and then kill that particular job https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnCommands.html ... View more

You're welcome, see a comment above from mclark. ... View more

oozie spark action is not supported in HDP 2.4 and earlier. We have it stated in release notes and Spark user guide. ... View more

Sorry, forgot to link https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_upgrading_hdp_manually/content/configure-hadoop-22.html ... View more

It says you already have a spark session. You don't need to set anything up just start your tutorial at the sparkpi example when your sandbox boots up. ... View more

You can't set number of mappers, it is determined by number of blocks in your dataset. The number of maps is usually driven by the total size of the inputs, that is, the total number of blocks of the input files. The right level of parallelism for maps seems to be around 10-100 maps per-node, although it has been set up to 300 maps for very cpu-light map tasks. Task setup takes a while, so it is best if the maps take at least a minute to execute. Thus, if you expect 10TB of input data and have a blocksize of 128MB, you’ll end up with 82,000 maps, unless Configuration.set(MRJobConfig.NUM_MAPS, int) (which only provides a hint to the framework) is used to set it even higher. https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html#Mapper ... View more

You are right, I will follow up to post the link to 2.3 ... View more

It was fixed in HDP 2.3.4 so yes it's in HDP 2.4 also http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_HDP_RelNotes/content/patch_hive.html ... View more