About ungarala

ungarala · ‎03-21-2016

Thanks for your response @Rich Raposa I eventually took a simpler approach to finish the exam (although 15 mins late 🙂 ) I am wondering why using MultipleInputs givesTaggedInputSplit object and not FileSplit.

ungarala · ‎03-21-2016

There is header in one of the CSV files and I was trying to ignore the first record. Below is my sample code. setup() { Path path = ((FileSplit) split).getPath(); filename = path.getName(); if (filename.equals("flightdata1.csv") { hasheader = true; } } map(....) { if (hasheader) { if (key.get() == 0) return; } ...... ...... }

ungarala · ‎03-20-2016

In practice exam I got classcast exception. Looks like TaggedInputSplit is not even a public class. How to get the filename when when using MultipleInputs? Mapper setup method: Path path = ((FileSplit) split).getPath(); Driver class: MultipleInputs.addInputPath(job, new Path(args[1]), TextInputFormat.class, FlightDataMapper.class); MultipleInputs.addInputPath(job, new Path(args[2]), TextInputFormat.class, WeatherDataMapper.class); Exception: java.lang.ClassCastException: org.apache.hadoop.mapreduce.lib.input.TaggedInputSplit cannot be cast to org.apache.hadoop.mapreduce.lib.input.FileSplit Any help is greatly appreciated. Thanks, Sanjay

ungarala · ‎03-16-2016

Thanks @Rich Raposa. It was actually little confusing to see ONLY setGroupingComparator mentioned in the objective, while secondary-sort involves writing comparator classes for sorting/grouping and using both setSortComparatorClass and setGroupingComparator methods.

ungarala · ‎03-16-2016

@Rich Raposa One of the objective in HDPCD:Java exam is to sort "output" of MR job using http://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/mapreduce/Job.html#setGroupingComparatorClass(java.lang.Class) My understanding is grouping comparator is for grouping records from multiple partitions. How can this be used for sorting? Do you mean using setSortComparatorClass? Thanks for your help!

ungarala · ‎03-16-2016

Thank you very much @Rich Raposa.

ungarala · ‎03-16-2016

There is an objective in HDPCD:Java exam to use LocaResource in MapReduce Jobs. I haven't come across any example on web to understand this. Appreciate if someone can give me pointers in understanding how LocalResource can be used in MR jobs? How is this different from cache files? http://hortonworks.com/training/class/hdp-certified-java-developer-exam/

Online	Offline
Last Visited	‎03-21-2016 05:24 PM

Member Since	‎03-16-2016 04:05 AM
Last Visited	‎03-21-2016 05:24 PM
Posts	7
Kudos received	8

Cloudera Community

Cloudera Community

Re: HDP Certified Java Developer practice exam

Re: HDP Certified Java Developer practice exam

HDP Certified Java Developer practice exam

Re: Sort the output of a MapReduce job

Sort the output of a MapReduce job

Re: Using a LocalResource instance to distribute f...

Using a LocalResource instance to distribute files...