Member since
04-11-2016
535
Posts
147
Kudos Received
77
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4096 | 09-17-2018 06:33 AM | |
891 | 08-29-2018 07:48 AM | |
1488 | 08-28-2018 12:38 PM | |
944 | 08-03-2018 05:42 AM | |
982 | 07-27-2018 04:00 PM |
07-05-2016
07:12 AM
1 Kudo
@Daniela Boamba Can you try running the insert query after changing execution engine to mr? set hive.execution.engine=mr; Thanks and Regards, Sindhu
... View more
07-01-2016
08:15 AM
1 Kudo
@Simran Kaur If you have a Hive metastore associated with your HDFS cluster, --hive-import and -hive-overwrite always writes to Hive warehouse directory. Arguments like --warehouse-dir <dir>, --as-avrodatafile, --as-sequencefile, --target-dir etc. are not honoured. Thanks and Regards, Sindhu
... View more
07-01-2016
07:38 AM
@Simran Kaur --target-dir is the while importing table data into HDFS using the Sqoop import tool and might not work with --hive-import. As @Dileep Kumar Chiguruvada explained earlier, the value of Hive warehouse directory will be picked from hive-site.xml. Thanks and Regards, Sindhu
... View more
07-01-2016
06:49 AM
1 Kudo
@Simran Kaur Can you please check the hive table created using describe formatted <hivetablename> and check the location of the hive data? It seems like data is being written to different directory and with --warehouse-dir not taking effect. Thanks and Regards, Sindhu
... View more
06-29-2016
10:46 AM
@Jan Kytara Can you please share the table definition?
... View more
06-28-2016
06:54 AM
@ARUNKUMAR RAMASAMY The jars need to be present on the Hiveserver machine. It is not necessary to have them on all the nodes. You can refer to below link: http://stackoverflow.com/questions/31802908/adding-hive-jars-permanently/31808016 Thanks and Regards, Sindhu
... View more
06-27-2016
11:39 AM
@Ethan Hsieh Have you run import-hive.sh? From where the tabled were created?
... View more
06-27-2016
09:39 AM
1 Kudo
@ARUNKUMAR RAMASAMY You can import data into underlying hbase table of phoenix, but data cannot be seen from phoenix. Sqoop Jira SQOOP-2649 for sqoop-phoenix integration is addressed on Sqoop 1.4.7. https://issues.apache.org/jira/browse/SQOOP-2649 Thanks and Regards, Sindhu
... View more
06-27-2016
04:44 AM
@Ethan Hsieh Issue seems to missing gson.jar in the AUX path. Please check and download the gson jar from the below link: http://www.java2s.com/Code/Jar/g/Downloadgson222jar.htm Hope this helps. Thanks and Regards, Sindhu
... View more
06-24-2016
01:15 PM
@jihed chokri Please try following: Login to Hive Cli. Run command set mapred.job.queue.name=<queue_name>; Run command set hive.execution.engine=mr; Run the query. Thanks and Regards, Sindhu
... View more
06-24-2016
10:54 AM
@jihed chokri Set the queue name in the hive cli and beeline as below: set set mapred.job.queue.name=<queue_name>; Or under hive-site.xml as property: <property>
<name>mapred.job.queue.name</name>
<value>queue_name</value>
</property> Thanks and Regards, Sindhu
... View more
06-23-2016
08:32 AM
@rahul jain You can use Hive view from Ambari and run queries on the Hive table. As first step, hive table needs to be created on top the HDFS file. Thanks and Regards, Sindhu
... View more
06-22-2016
01:42 PM
2 Kudos
@alain TSAFACK You can load the data from csv file to a temp hive table with same structure as orc table, then insert the data into orc table as: insert into table table_orc as select * from table_textfile; Thanks and Regards, Sindhu
... View more
06-21-2016
10:05 AM
2 Kudos
@Michel Sumbul CBO is mainly for optimization decisions which reduces the cost of query execution and is independent of storage formats like ORC. Below is some of decisions based on CBO:
How to order Join What algorithm to use for a given Join Should the intermediate result be persisted or should it be recomputed on operator failure. The degree of parallelism at any operator (specifically number of reducers to use). Semi Join selection For details, please refer to below link: https://cwiki.apache.org/confluence/display/Hive/Cost-based+optimization+in+Hive Thanks and Regards, Sindhu
... View more
06-21-2016
07:16 AM
2 Kudos
@ARUNKUMAR RAMASAMY MySQL might be rejecting connections to the extract the data from the tables from remote host. We need to grant privileges for the IP's of the data nodes at the database end as below: GRANTALL PRIVILEGES ON*.*TO'user'@'ipadress' Thanks and Regards, Sindhu
... View more
06-21-2016
06:43 AM
@ARUNKUMAR RAMASAMY The communication between the datanodes and mysql needs to be open. Make sure telnet <mysql_server> <port> works on all the nodes in the cluster. Also, need to verify the bind address at mysql end to verify the connectivity. You can refer to below link for more debugging at mysql end: http://stackoverflow.com/questions/2121829/com-mysql-jdbc-exceptions-jdbc4-communicationsexceptioncommunications-link-fail Hope this helps. Thanks and Regards, Sindhu
... View more
06-20-2016
09:17 AM
1 Kudo
@Pradeep Bhadani Run mysql manual install on the machine itself as: rpm -e --nodeps mysql-libs Hope this helps. Thanks and Regards, Sindhu
... View more
06-20-2016
06:14 AM
Could you please share the steps that resolved the issue and mark as best answer? Thanks, Sindhu
... View more
06-17-2016
06:51 AM
1 Kudo
@Simran Kaur Seems like TotalRecords is a keyword. Try using TotalRecords_1 and see if it helps. Thanks and Regards, Sindhu
... View more
06-16-2016
04:59 PM
1 Kudo
@khushi kalra You can also use RJDBC as below to connect to Hive: library("DBI") library("rJava") library("RJDBC") hive.class.path = list.files(path=c("/usr/hdp/current/hive-client/lib"), pattern="jar", full.names=T); hadoop.lib.path = list.files(path=c("/usr/hdp/current/hive-client/lib"), pattern="jar", full.names=T); hadoop.class.path = list.files(path=c("/usr/hdp/2.4.0.0-169/hadoop"), pattern="jar", full.names=T); cp = c(hive.class.path, hadoop.lib.path, hadoop.class.path, "/usr/hdp/2.4.0.0-169/hadoop-mapreduce/hadoop-mapreduce-client-core.jar") .jinit(classpath=cp) drv <- JDBC("org.apache.hive.jdbc.HiveDriver","hive-jdbc.jar",identifier.quote="`") url.dbc <- paste0("jdbc:hive2://ironhide.hdp.local:10000/default"); conn <- dbConnect(drv, url.dbc, "hive", “redhat"); log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. dbListTables(conn); Thanks and Regards, Sindhu
... View more
06-16-2016
07:43 AM
2 Kudos
@Roberto Sancho Please refer to below Hortonworks blog with inputs to improve Hive query performance: http://hortonworks.com/blog/5-ways-make-hive-queries-run-faster/ Hope this helps. Thanks and Regards, Sindhu
... View more
06-15-2016
06:18 AM
2 Kudos
@Ethan Hsieh
Looks like hive metastore is using mysql in your case, add the mysql client jar to <atlas package>/bridge/hive/. That should work. Ideally, import-hive.sh should use hive classpath so that all hive dependencies are included. Currently, we bundle hive dependencies as well and hence this issue if hive uses non-default driver. Details: https://issues.apache.org/jira/browse/ATLAS-96 Hope this helps. Thanks and Regards, Sindhu
... View more
06-14-2016
02:14 PM
2 Kudos
@Venkat Chinnari The issue seems to be with cast from text to parquet. Try creating a sample table say table3 without serde properties but just 'stored as parquet' and check if insert overwrite works. Thanks and Regards, Sindhu
... View more
06-14-2016
12:07 PM
5 Kudos
@Shihab The temp tables are created during the application run as intermediate data. These intermediate tables will not be removed in case the application fails and cleanup does not happen. Please check if applications are running which is generating data. Meanwhile, you can also try compressing the intermediate data by setting the property "hive.exec.compress.intermediate" as true in hive-site.xml. The related compression codec and other options are determined from Hadoop configuration variables mapred.output.compress*. Hope this helps. Thanks and Regards, Sindhu
... View more
06-10-2016
10:39 AM
@Tajinderpal Singh You can refer to below Spark documentation: http://spark.apache.org/docs/latest/streaming-kafka-integration.html Thanks and Regards, Sindhu
... View more
06-10-2016
10:29 AM
@a kumar This can be relate to Hive Jira 12349:
https://issues.apache.org/jira/browse/HIVE-12349 Can you please share the query being run? Thanks and Regards, Sindhu
... View more
06-10-2016
10:26 AM
@Varun Kumar Chepuri To initialize Metastore is to initialize the metastore database. Refer to the below link for manual configuration: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_installing_manually_book/content/set_up_hive_hcat_configuration_files.html Hope this helps. Thanks and Regards, Sindhu
... View more
06-10-2016
09:15 AM
1 Kudo
@alain TSAFACK You can also make use of --query option during sqoop import to cast the smalldatetime to timestamp: sqoop import ...other options.... --query "select cast(col1 as datetime) from table_name" Hope this helps. Thanks and Regards, Sindhu
... View more
- « Previous
- Next »