Hi Joe, Please refer following useful links for answer. What's the advantage of HDF Kerberisation ? http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_security/content/ch_hdp-security-guide-overview.html is it only for Kafka and Storm ? https://hortonworks.com/blog/enabling-kerberos-hdp-active-directory-integration/ https://hortonworks.com/blog/office-hours-qa-on-yarn-in-hadoop-2/ Hope this will be useful. ... View more

Hi Jose, From error log message I found that it's because of checksum mismatch. Please refer below links. Hope it will work. 1. http://stackoverflow.com/questions/15434709/checksum-exception-when-reading-from-or-copying-to-hdfs-in-apache-hadoop 2. https://issues.apache.org/jira/browse/HDFS-6804 Thanks, Mahesh ... View more

Hi Varma, Please go through below links. Hope it will be useful. http://stackoverflow.com/questions/28119423/standard-practices-for-logging-in-mapreduce-jobs http://grepalex.com/2012/11/12/hadoop-logging/ Thanks, Mahesh ... View more

Hi Yogesh, Please refer below links. It will be useful for you. http://stackoverflow.com/questions/25257102/how-to-store-binary-data-in-hive https://community.hortonworks.com/questions/10764/effective-way-to-store-image-files-pdf-files-in-hd.html https://community.hortonworks.com/questions/4024/how-many-files-is-too-many-on-a-modern-hdp-cluster.html ... View more

Hi Aparna, Please go through this URL hope it will help you. http://stackoverflow.com/questions/25242287/filenotfoundexception-file-too-large ... View more

Hi Amith, Please refer this video he has explained very well MapReduce flow chart. Hope this will be useful. https://www.youtube.com/watch?v=6OemZEJdMp8 Thanks, Mahesh ... View more

Hi Vamsi, I have found some simple articles on this topic please go through following links. Hope it will be useful for you. http://www.ericlin.me/how-to-use-hive-lateral-view-in-your-query http://stackoverflow.com/questions/20667473/hive-explode-lateral-view-multiple-arrays http://stackoverflow.com/questions/33993123/hive-lateral-view-with-sample-example-with-hive-table-having-1-column-as-array Thanks, Mahesh ... View more

HI, Please refer below example. With a little bash we can loop through all our tables, appending the results to a file: #get a list of the files hive -e "show tables;" > hivetables.txt Now — cool stuff: for line in $(cat hivetables.txt) ; do results=$(hive -hiveconf tablename=$line -f test.hql) echo $results >> tablecounts.txt done Hope this will be helpful. Thanks, Mahesh ... View more

@Amit Dass, first create table in Hive inside database and that table name should match with TBLPROPERTIES table name. ... View more

Hi Vaibhav, Please go through this link https://cwiki.apache.org/confluence/display/Hive/Enhanced+Aggregation,+Cube,+Grouping+and+Rollup ... View more

Thank You! ... View more

Thank you! ... View more

Hi Arkaprova, Every programming languages have their own purpose and advantages are there ex. COBOL (Common Bussiness Oriented Language) used for bussiness oriented purposes, FORTRAN (Formula Translation) using in scientific research puposes etc. Big Data domain again has few specific fields are there like BigData developer, BigData analytics, Data Scientist etc. Based on your interest in these fields you can choose language for Bigdata developer you need to learn stuffs like Hadoop, MapReduce, Pig, Hive, Flume, Sqoop etc for analytics you need to learn Python or Scala, SAS etc. so depending on fields these technologies will matter. Hope this will give at least some basic idea about Big Data world. ... View more

Hi Michael, Thank for sharing useful links. ... View more

Hi Arun, If you are using java to write UDF please follow the below steps it will be easy and helpful. To write UDF you have to follow three main steps. Write UDF program and compile it and create jar file. Register that .jar file using Register command in Pig. Define .jar files in your Pig script using Define command. public class Sample_Eval extends EvalFunc<String>{ public String exec(Tuple input) throws IOException { if (input == null || input.size() == 0) return null; String str = (String)input.get(0); return str.toUpperCase(); } } This is just a sample exaple of UDF program in java. In the above example, I have return the code to convert the contents of the given column to uppercase. Like this you can write your own UDF just using your custom code. ... View more