Member since
08-23-2016
62
Posts
44
Kudos Received
8
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
16722 | 11-13-2016 06:24 PM | |
394 | 11-13-2016 04:21 PM | |
1051 | 11-13-2016 03:58 PM | |
5907 | 10-05-2016 04:45 AM | |
1067 | 09-29-2016 06:42 AM |
03-13-2017
01:03 PM
Hi Joe, Please refer following useful links for answer. What's the advantage of HDF Kerberisation ? http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_security/content/ch_hdp-security-guide-overview.html is it only for Kafka and Storm ? https://hortonworks.com/blog/enabling-kerberos-hdp-active-directory-integration/ https://hortonworks.com/blog/office-hours-qa-on-yarn-in-hadoop-2/ Hope this will be useful.
... View more
03-02-2017
03:05 PM
Hi Jose, From error log message I found that it's because of checksum mismatch. Please refer below links. Hope it will work. 1. http://stackoverflow.com/questions/15434709/checksum-exception-when-reading-from-or-copying-to-hdfs-in-apache-hadoop 2. https://issues.apache.org/jira/browse/HDFS-6804 Thanks, Mahesh
... View more
11-25-2016
07:58 AM
1 Kudo
Hi Varma, Please go through below links. Hope it will be useful. http://stackoverflow.com/questions/28119423/standard-practices-for-logging-in-mapreduce-jobs http://grepalex.com/2012/11/12/hadoop-logging/ Thanks, Mahesh
... View more
11-23-2016
09:55 AM
Hi Yogesh, Please refer below links. It will be useful for you. http://stackoverflow.com/questions/25257102/how-to-store-binary-data-in-hive https://community.hortonworks.com/questions/10764/effective-way-to-store-image-files-pdf-files-in-hd.html https://community.hortonworks.com/questions/4024/how-many-files-is-too-many-on-a-modern-hdp-cluster.html
... View more
11-17-2016
01:11 PM
Hi Aparna, Please go through this URL hope it will help you. http://stackoverflow.com/questions/25242287/filenotfoundexception-file-too-large
... View more
11-14-2016
03:53 PM
Hi Amith, Please refer this video he has explained very well MapReduce flow chart. Hope this will be useful. https://www.youtube.com/watch?v=6OemZEJdMp8 Thanks, Mahesh
... View more
11-13-2016
06:24 PM
Hi Kumar, Please check this https://issues.apache.org/jira/browse/HIVE-10294
... View more
11-13-2016
04:21 PM
1 Kudo
Hi Roger, Please refer this URL they have discussed same thing in this...it will be useful for you http://tecadmin.net/set-up-hadoop-multi-node-cluster-on-centos-redhat/#
... View more
11-13-2016
03:58 PM
1 Kudo
Hi, Please refer this link. Hope it will be useful. http://stackoverflow.com/questions/38402140/how-to-import-data-from-mysql-into-hive-using-apache-nifi
... View more
11-09-2016
01:26 PM
1 Kudo
Hi Vamsi, I have found some simple articles on this topic please go through following links. Hope it will be useful for you. http://www.ericlin.me/how-to-use-hive-lateral-view-in-your-query http://stackoverflow.com/questions/20667473/hive-explode-lateral-view-multiple-arrays http://stackoverflow.com/questions/33993123/hive-lateral-view-with-sample-example-with-hive-table-having-1-column-as-array Thanks, Mahesh
... View more
10-21-2016
06:16 AM
4 Kudos
HI, Please refer below example. With a little bash we can loop through all our tables, appending the results to a file:
#get a list of the files hive -e "show tables;" > hivetables.txt Now — cool stuff: for line in $(cat hivetables.txt) ;
do
results=$(hive -hiveconf tablename=$line -f test.hql)
echo $results >> tablecounts.txt done Hope this will be helpful. Thanks, Mahesh
... View more
10-05-2016
02:25 PM
1 Kudo
@Amit Dass, first create table in Hive inside database and that table name should match with TBLPROPERTIES table name.
... View more
10-05-2016
04:45 AM
2 Kudos
Hi Anand, Please refer these links. I hope it will be helpful. https://community.mapr.com/docs/DOC-1215 http://stackoverflow.com/questions/37774532/unable-to-start-zookeeper
... View more
10-04-2016
01:36 PM
3 Kudos
Hi, I'm trying to load simple dataset into HBase using Pig script. I have referred few websites but some are using org.apache.pig.backend.hadoop.hbase.HBaseStorage and in som website they used org.apache.hadoop.hive.hbase.HBaseStorageHandler. Can someone please let me know which is the correct method and what is the difference between these two.?
... View more
Labels:
- Labels:
-
Apache HBase
-
Apache Hive
-
Apache Pig
10-04-2016
07:34 AM
1 Kudo
Hi Vaibhav, Please go through this link https://cwiki.apache.org/confluence/display/Hive/Enhanced+Aggregation,+Cube,+Grouping+and+Rollup
... View more
09-29-2016
06:55 AM
09-29-2016
06:42 AM
1 Kudo
Hi Jyothi, please refer this link hope this will be helpful for you. http://stackoverflow.com/questions/17533661/zookeeper-error-connection-loss-exception
... View more
09-28-2016
10:19 AM
1 Kudo
Hi Rohini, along with this link http://hortonworks.com/hadoop-tutorial/learning-the-ropes-of-the-hortonworks-sandbox you can also refer Definitive Guide 4th edition book. In this book author explained very well about all basic Hadoop technologies.
... View more
09-23-2016
06:53 PM
1 Kudo
Hi Arkaprova, Every programming languages have their own purpose and advantages are there ex. COBOL (Common Bussiness Oriented Language) used for bussiness oriented purposes, FORTRAN (Formula Translation) using in scientific research puposes etc. Big Data domain again has few specific fields are there like BigData developer, BigData analytics, Data Scientist etc. Based on your interest in these fields you can choose language for Bigdata developer you need to learn stuffs like Hadoop, MapReduce, Pig, Hive, Flume, Sqoop etc for analytics you need to learn Python or Scala, SAS etc. so depending on fields these technologies will matter. Hope this will give at least some basic idea about Big Data world.
... View more
09-23-2016
05:57 PM
2 Kudos
Hi Kishor, Please refer this link to setup multi agent setup hope this will be helpful for you http://hadooptutorial.info/multi-agent-setup-in-flume/
... View more
09-22-2016
01:38 PM
1 Kudo
Hi Ashnee, There are some known issues and they are listed in the following link please refer this hope this will be helpful for you. https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_releasenotes_ambari_2.1.0.0/content/ambari_relnotes-2.1.0.0-known-issues.html Thanks, Mahesh
... View more
09-08-2016
06:09 AM
Hi Michael, Thank for sharing useful links.
... View more
09-08-2016
06:07 AM
3 Kudos
Hi Arun, If you are using java to write UDF please follow the below steps it will be easy and helpful. To write UDF you have to follow three main steps. Write UDF program and compile it and create jar file. Register that .jar file using Register command in Pig. Define .jar files in your Pig script using Define command. public class Sample_Eval extends EvalFunc<String>{
public String exec(Tuple input) throws IOException {
if (input == null || input.size() == 0)
return null;
String str = (String)input.get(0);
return str.toUpperCase();
}
} This is just a sample exaple of UDF program in java. In the above example, I have return the code to convert the contents of the given column to uppercase. Like this you can write your own UDF just using your custom code.
... View more