On the exam you should always use Ambari when possible, especially for tasks like enabling NameNode HA. ... View more

@John Cod - Try adding the following environment variables to spark-env.sh (found in the /conf folder of your Spark install) - using the appropriate IP address of course if Spark is running on a machine other than localhost: export SPARK_MASTER_IP=127.0.0.1 export SPARK_LOCAL_IP=127.0.0.1 ... View more

@Anshu Kumar - this is not a concern that you need to have on the exam. In order for a task on the exam to be marked as correct, you must have already executed your code and the output must exist in HDFS. If the desired output folder specified by the task instructions is not in HDFS, then that task will be immediately marked as wrong. Do not worry about how the evaluator grades the exam. Focus on generating the desired output by carefully following the task instructions, making sure that you run your code, and making sure your code generated the desired output. ... View more

An easier solution would be to read every row, and then add a check to see if it's a header row by checking one of the columns that you know only appears in a header row. ... View more

Sorting in MR applies to two areas: Sort output by keys: this done "naturally" in the sense that the keys are sorted as they come into the reducer. The compareTo method in the key class determines this natural sorting Secondary sort: both the keys and values are sorted. That involves writing a group comparator class and then registering that class with the MR Job using the setGroupingComparator class The exam objective you listed above is referring to both. The first one is fairly straightforward - you implement the compareTo method in your key class. The secondary sort involves a bit more work. There is a nice blog here that has an example of how to implement a secondary sort: https://vangjee.wordpress.com/2012/03/20/secondary-sorting-aka-sorting-values-in-hadoops-mapreduce-programming-paradigm/ ... View more

@Sanjay Ungarala - thank you for pointing this out. We decided a while back to remove this objective from the HDPCD:Java exam, but the website was not changed accordingly. I just updated the website and removed that objective. LocalResource is used heavily in YARN, but it is beyond the scope of our Java exam. ... View more

@Mahesh Deshmukh I'm not sure what your need is, but null values should be filtered first. The general rule of thumb in Pig is to "filter early and often" to minimize the amount of data that gets shuffled and sorted, so before the foreach: For example: a = "some Pig relation" b = filter a by $1 is not null; //filter out tuples where the $1 field is null c = foreach b generate ... //no need to worry about $1 being null The term "empty" refers to bags typically, and in particular you can use the isEmpty function to check if a bag is empty. You normally do this after a GROUP command: a = "some Pig relation" b = group a by $3; c = filter b by not IsEmpty(group); What are you trying to accomplish? ... View more

Many of the Pig operators have a PARALLEL option for specifying the number of reducers, which also determines the number of output files. For the intent of the practice exam and the real exam, using PARALLEL is all you need to accomplish this task. ... View more

@Gurjinder Singh - to fully be prepared for the exam, you should be able to perform all of the exam objectives listed here: http://hortonworks.com/training/class/hdp-certified-developer-hdpcd-exam/ I highly recommend also taking the practice exam. The details are on that webpage above. It uses an environment similar to the real exam and contains tasks similar to the real exam, and it's a great way to test your knowledge and see how ready you are for the exam. Keep in mind the practice exam does not cover every exam objective, so use it only as an aid to prepare. Like I said first - to be fully prepared make sure you can perform all of the exam objectives listed on our website. If you need coding experience and are looking for ideas, check out the tutorials on our website: http://hortonworks.com/tutorials/ ... View more

@Gurjinder Singh Make sure you have a table in Hive named 'emp1' in the 'default' database. ... View more

@Saurabh Singh The real environment looks the same as the practice environment The tasks are displayed differently though - on the real exam, the tasks appear in a separate pop-up window that you can move around Each task will tell you where to save your answer. For example, Task 1 might tell you to save your script in a file named /home/horton/solutions/task1 The answers are evaluated by the output. There are always different ways to solve problems in programming. It is not the code we are interested in - it's the result. You need to execute your code on the real exam and it needs to generate the desired output described in the task instructions. (If a task has no output, it is immediately marked as wrong.) Each task is worth 1 point and it is either correct or wrong - there is no partial credit. The current HDPCD exam has 7 tasks and you need to get 5 correct to pass. (This is subject to change at any time.) ... View more

To fully prepare for an exam, I recommend two areas of focus: You should be able to perform all of the exam objectives listed on our website You should attempt the corresponding practice exam Taking the practice exam alone is not enough - it is only meant to give you an idea of what the exam environment is like and what the tasks are like. To be fully prepared, you should be able to perform every task listed in the exam objectives. Good idea! I like the idea of collaborating and sharing ideas for exam prep, as long as everyone is careful not to share actual exam details - which would be a violation of the candidate agreement. The best way to collaborate would be here on the Hortonworks Community Connection and tagging all posts with "HDPCA". I will start tagging exam posts as they come in. ... View more

Good question. If you look at the current exam description, it uses Ambari 1.7 which does not have Views for Pig and Hive: http://hortonworks.com/training/class/hdp-certified-developer-hdpcd-exam/ However, the HDPCD exam is about to be updated to HDP 2.3 with Ambari 2.2, so anyone preparing for our exams - make sure you know which version of Ambari and HDP your exam will be on. This information will always be available on our website: http://hortonworks.com/training/certification/ That being said, your exam is graded by the output of your code, so it is irrelevant how you write the code. You can use any text editor and run the scripts in any way you choose. ... View more

Hi @Raghunadh Chellinki Sorry about the confusion. The option for "ec2-classic" is only available if your Amazon account is more than 2 years old. If you had to create a new AWS account, then you will have to launch your instance into a new VPC. Just make sure you enable "Auto-assign Public IP address" - then everything will work fine from there. Let me know if you have any other questions. Thanks, Rich ... View more