Member since
09-24-2015
816
Posts
488
Kudos Received
189
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2657 | 12-25-2018 10:42 PM | |
12157 | 10-09-2018 03:52 AM | |
4193 | 02-23-2018 11:46 PM | |
1882 | 09-02-2017 01:49 AM | |
2197 | 06-21-2017 12:06 AM |
03-08-2017
03:09 AM
1 Kudo
Sorry for the delay, the best way to check your queries provided it's Hive on Tez, which is pretty much standard these days since it's much faster than Hive on MR, is through Ambari Tez view. You can see the query, start and end time, and DAG details, counters, graphical view of used mappers and reducers, indeed very detailed information. Note that if you run Pig on Tez those queries will appear in the Tez view as well. Give it a try!
... View more
03-06-2017
07:31 AM
What's the output of "hostname" and "hostname -f" commands on your host where the error occurs? Also, cd to /etc/security/keytabs and run: "klist -kt hive.service.keytab", I suspect the principal stored in the keytab is wrong.
... View more
03-05-2017
09:32 AM
Great! Please up-vote and/or accept the answer which helped you. Tnx!
... View more
03-05-2017
07:28 AM
You have maybe uncovered a bug in Ambari-2.4.2. There is a file hivemetastore-site.xml in HDP-2.5.x but not in HDP-2.4.x which you are using. I'm enclosing my hivemetastore-site.xml, put it in your /etc/hive/conf/conf.server directory on the node where HS2 and Hive-metastore are running, and then try to restart Hive metastore and HS2. If all this doesn't work you may consider to remove all HDP binaries and install HDP-2.5.3 with your Ambari-2.4.2hivemetastore-site.xml
... View more
03-05-2017
01:36 AM
Your "credential" section looks wrong, it should be something like this: <property>
<name>hcat.metastore.uri</name>
<value>thrift://<host>:<port></value>
</property>
<property>
<name>hcat.metastore.principal</name>
<value>hive/<host>@<realm></value>
</property> On every node where Oozie client is installed you can find good examples for all Oozie actions including Hive action in "/usr/hdp/current/oozie-client/doc/examples". Check file called apps/hive/workflow.xml.security under "examples" and modify job.properties to provide your "realm" and other required parameters. Also, in case of hive2 action be sure to test using HS2 server running in binary transport mode. There were some bugs in http mode on kerberized cluster. This applies only to hive2 action, the hive action you are trying should work on both transport modes.
... View more
03-05-2017
01:18 AM
2 Kudos
Well, there is no way to know in advance how long will a query you've never tried before take on your cluster. Besides query complexity and resource requirements it also depends on the size and occupancy of the Yarn queue you are using. Even the same query can take longer the first time to start Tez containers if not already started, or to load data to memory in case of LLAP. You can try to run your representative queries and measure their execution time, after that you will roughly know how long are queries from the same class going to take (again if Yarn queue conditions are about the same).
... View more
03-05-2017
12:58 AM
Hi @vamsi valiveti
I noticed that you ask a lot of questions but haven't accepted many answers in the last few months. So, if you don't mind let me tell you a few words how this work: Both questions and answers can be up-voted if the others find them helpful. Also, for each question one answer can be "accepted", usually by the user who asked the question, if that answer resolved the question or greatly helped the user to resolve the issue by himself. Accepted answers help HCC to better manage answered questions, and also help the others who search and find that question later to know that the accepted answer indeed works and can be trusted. So, it will be great if you accept one answer to this question, and some other questions you recently asked. Many thanks!
... View more
03-04-2017
04:45 AM
1 Kudo
Check your livy.superusers in Spark->Livy in Ambari, it should mach your Zeppelin principal, in this case zeppelin-lake. You are using custom zeppelin service name, there may be also some bugs in that case. Also check your Zeppelin principal stored in zeppelin.server.kerberos.keytab. Many other issues are covered in this post and it this blog, but Zeppelin service name and the principal in those posts is "zeppelin".
... View more
03-04-2017
04:22 AM
2 Kudos
The output of STRSPLIT is a tuple, so if you want to provide its schema you need to explicitly say for example "t1:tuple", like below, and after that you can refer to it as t1.a1 and t1.a1of1. With FLATTEN you get rid of the tuple. So you can choose which way to declare it. grunt> b = FOREACH a generate STRSPLIT(a1,'\\u002E') as (t1:tuple(a1:chararray,a1of1:chararray)), a2, a3;
grunt> describe b;
b: {t1: (a1: chararray,a1of1: chararray),a2: chararray,a3: chararray}
grunt> c = foreach b generate t1.a1, a3;
... View more
03-04-2017
02:12 AM
You can export OOZIE_TIMEZONE the same way you export OOZIE_URL, or use "-timezone" option, or for a long-term change set "oozie.processing.timezone" in oozie-site.xml for example to "GMT+0900".
... View more