Member since
07-07-2016
11
Posts
2
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3061 | 10-02-2016 10:24 PM | |
847 | 09-15-2016 12:47 PM |
10-04-2016
03:29 PM
Also, for posterity... to make this easier to manage, when using the built-in Phoenix, one can simply use the path to the phoenix-hive jar installed by HDP. So the path becomes export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH:/usr/hdp/current/phoenix-client/phoenix-hive.jar This is better, when using HDP/Ambari managed Phoenix since you won't need install new jars, or change this config if/when you upgrade Phoenix. Just bounce HiveServer2 as needed...
... View more
10-03-2016
05:18 PM
Follow up (for posterity): beeline does work with the storage handlers. I found that the changes I made to the hive aux path were being undone. To fix this, I installed the phoenix-hive.jar on all nodes (workers and client/edge nodes) add added the following to "hive-env template" in Ambari (which, bounces HiveServer2 to make it take effect) export
HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH:/opt/hive/auxpath/phoenix-hive.jar
... View more
10-03-2016
01:55 PM
If anyone has experience with the HiveStorage handler and has an opinion on whether it works with beeline, I'd appreciate your thoughts.
... View more
10-02-2016
10:24 PM
1 Kudo
I found the problem.... If I run the exact same commands with the "real" hive CLI command, it works. If I use beeline, it fails. This is troubling because "add jars" was done using an HDFS path, so the environment should be the same. Worse, the hive CLI is headed for removal if/when beeline can take over. Let's hope that doesn't happen too soon.
... View more
10-02-2016
09:57 PM
I'm trying to use the Hive StorageHandler for Pheonix, but getting this error: Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:No suitable driver found for jdbc:phoenix:ip-10-113-98-203.ec2.internal:2181:/hbase-unsecure;) (state=08S01,code=1) I know that Hive and Phoenix are working on the cluster (which is HDP2.5). This test shows that DNS is working, a valid Zookeeper server, and the ZNode for Hbase zookeeper-client -server ip-10-113-98-203.ec2.internal:2181 ls / | tail -1 [registry, hiveserver2, zookeeper, hbase-unsecure, rmstore] I add the jars in the hive script like this: -- hadoop fs -copyFromLocal /usr/hdp/current/phoenix-client/phoenix-hive.jar /user/etl/hive_libs ADD JAR hdfs:///user/etl/hive_libs/phoenix-hive.jar; ADD JAR hdfs:///user/etl/hive_libs/phoenix-client.jar; Using this example (https://phoenix.apache.org/hive_storage_handler.html) and replacing the the hostname and ZNode per above gives the command that fails. createtablephoenix_table ( s1 string, i1 int, f1 float, d1 double ) STORED BY'org.apache.phoenix.hive.PhoenixStorageHandler' TBLPROPERTIES ( "phoenix.table.name" = "phoenix_table", "phoenix.zookeeper.quorum" = "ip-10-113-98-203.ec2.internal", "phoenix.zookeeper.znode.parent" = "/hbase-unsecure", "phoenix.zookeeper.client.port" = "2181", "phoenix.rowkeys" = "s1, i1", "phoenix.column.mapping" = "s1:s1, i1:i1, f1:f1, d1:d1", "phoenix.table.options" = "SALT_BUCKETS=10, DATA_BLOCK_ENCODING='DIFF'" ); Does anyone who uses the Hive StorageHandler have an idea about this?
... View more
Labels:
- Labels:
-
Apache Phoenix
09-15-2016
12:47 PM
As a workaround, I have built a script to get the correct jars from a cluster and install them as local maven artifacts.
... View more
09-14-2016
02:53 PM
1 Kudo
We have an HDP2.4 cluster (provisioned with CloudBreak), which has Map/Reduce version 2.7.1.2.4.3.0-225. For example, the cluster has this file: /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-client-common-2.7.1.2.4.3.0-225.jar But, the latest M/R version is 2.7.1.2.4.2.8-3 (and then it jumps to 2.7.3.2.5.0.0-1245), according to Sonatype index and the underlying HTTP server http://repo.hortonworks.com/content/repositories/releases/org/apache/hadoop/hadoop-mapreduce-client-app/ http://repo.hortonworks.com/index.html#nexus-search;gav~org.apache.hadoop~hadoop-mapreduce-examples~~~~kw,versionexpand So, my question is: how do I get the correct artifact for the HDP distro I have?
... View more
Labels:
- Labels:
-
Apache Hadoop