Hi, I am following Spark 1.6 tech preview tutorial as mentioned in the link http://hortonworks.com/hadoop-tutorial/apache-spark-1-6-technical-preview-with-hdp-2-3/ In one of the steps in mentions to switch to spark OS user by doing su spark. Just wondering if there is any alternate way of running spark applications by proxy user. For example, if I am logging into the machine as user 'tom', who doesn't have sudo rights, then how to run spark applications through the shell without changing to spark user. Many Thanks in advance ... View more

Are there any recommendations / guidelines / best practices from Hortonworks or any other Hadoop provider in order to setup and maintain a cluster of any size? Thanks in advance ... View more

Hi, I am trying to run TestDFSIO performance tests on my cluster and trying to do as below. Logged into the name node machine Logged in as hdfs user /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-*tests.jar TestDFSIO -write -nrFiles 100 -fileSize 100 TestDFSIO Read Test hadoop jar After running the above test, I am seeing a permission denied error. Not sure what could be the reason for it as I am running as hdfs user, but still it doesnt run. bash: /usr/hdp/2.3.4.0-3485/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.7.1.2.3.4.0-3485-tests.jar: Permission denied Any idea what could be the reason? Thanks in advance. ... View more