Re: Why Hive Compaction is failing ?

rajkumar_singh · ‎02-14-2020

@Ellyly the NPE you got is side effect of Exception you have in stage-3/4/5, can you please upload the Error stack you got before the NPE, I am assuming that you are running hive on tez? I think ReExecOverlayPlugin failing with NPE and shallowing the exception at client-side (fix for this has been identified as Hive-19204 but you should be able to see the real cause in HS2 logs

rajkumar_singh · ‎02-12-2020

I think error log is pretty informative, getGroups call for $Superuser does not get any group associated with the user, you might need to run "hdfs groups" command on the HMS node for this user if that the case and fix the env issue.

rajkumar_singh · ‎02-12-2020

@Ellyly can you please post the complete stack trace from the HiveServer2 logs?

rajkumar_singh · ‎12-14-2017

if you are running query on tez then this will be useful to follow https://community.hortonworks.com/articles/22419/hive-on-tez-performance-tuning-determining-reducer.html

rajkumar_singh · ‎08-10-2017

add following in connection string tez.queue.name=<queue_name>

rajkumar_singh · ‎08-02-2017

Creating and running Temporary functions are discouraged while running a query on LLAP because of security reason since many users are sharing same instances of LLAP, it can create a conflict but still, you can create temp functions using add jar and hive.llap.execution.mode=auto. with exclusive llap execution mode(hive.llap.execution.mode=only) you will run into the ClassNotFoundException, hive.llap.execution.mode=auto will allow some part of query(map tasks) to run in the tez container. Here are steps to create a custom permanent function in LLAP(steps are tested on HDP-260) 1. create a jar for UDF function (in this case I am using simple udf): git clone https://github.com/rajkrrsingh/SampleCode mvn clean package 2. upload the target/SampleCode.jar to the node where HSI is running(in my case I have copied it to /tmp directory) 3. add jar to hive_aux_jars (goto Ambari--> hive --> config --> hive-interactive-env template) export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH:/tmp/SampleCode.jar 4. add the jar to Auxillary JAR list (goto Ambari--> hive --> config --> Auxillary JAR list) Auxillary JAR list=/tmp/SampleCode.jar 5. restart LLAP 6. create Permanent Custom function connect to HSI using beeline create FUNCTION CustomLength as 'com.rajkrrsingh.hiveudf.CustomLength'; describe function CustomLength; select CustomLength(description) from sample_07 limit 1; 7. check where the SampleCode.jar localized root@hdp26 container_e06_1501140901077_0019_01_000002]# pwd /hadoop/yarn/local/usercache/hive/appcache/application_1501140901077_0019/container_e06_1501140901077_0019_01_000002 [root@hdp26 container_e06_1501140901077_0019_01_000002]# find . -iname sample* ./app/install/lib/SampleCode.jar

rajkumar_singh · ‎07-28-2017

@Anurag Mishra this is sample curl request to get information of zookeeper service, hope this will serve your purpose curl -u admin:admin -H "X-Requested-By: ambari" -X GET http://<AMBARI_HOST>:8080/api/v1/clusters/<CLUSTERNAME>/services/ZOOKEEPER

rajkumar_singh · ‎07-28-2017

@suresh krish check if HSI is up and running and make sure you have nothing wrong with zookeeper.

rajkumar_singh · ‎07-28-2017

@mungeol heo you can change the logging level of hiveserver2 using ambari, go to hive--> config --> Advanced hive-log4j --> replace hive.root.logger=INFO,DRFA with hive.root.logger=DEBUG,DRFA and restart hiveserver2

rajkumar_singh · ‎07-19-2017

hive locking will effect on non-transcational table as well while these setting turned on.

rajkumar_singh · ‎07-19-2017

@sundaramoorthy muthusamy it will depend on whether you have enabled concurrency or not (hive.support.concurrency), with concurrency enabled and txn manager set as DbTxManager, while the query is running it will take a write lock on table and once you fire alter table set partition location, this DDL statement will wait for query to complete and release lock to perform alter operation or will time out (controlled by hive.txn.timeout)

rajkumar_singh · ‎07-19-2017

@Gagan Brahmi modify tez.am.resource.memory.mb in ambari tez configuration, restart tez and hive client.

rajkumar_singh · ‎07-17-2017

@suresh krish it seems that it is taking PtyCryptoCodec from some configuration, you need to check from where it is taking this configuration, run hive -e 'set -v;' > hive.out and grep codec in hive.out to know from where it is picking the PtyCryptoCodec

rajkumar_singh · ‎07-17-2017

@sathish jeganathan can you get the output of following hive --hiveconf hive.root.logger=DEBUG,console > hivecli.out 2>&1

rajkumar_singh · ‎07-17-2017

@suresh krish what the value for key io.compression.codecs in hdfs config using ambari?

rajkumar_singh · ‎07-17-2017

@suresh krish can you add the output of following command hive --hiveconf hive.root.logger=DEBUG,console -e "show databases;"

rajkumar_singh · ‎07-17-2017

@suresh krish then Atlas hook is not an issue here, do you find any exception/error in metastore logs?

rajkumar_singh · ‎07-17-2017

@sathish jeganathan could you please post hivecli.out after running following command hive > hivecli.out 2>&1

rajkumar_singh · ‎07-17-2017

@suresh krish can you check what the value for the hive.exec.post.hooks in hive config using ambari?

rajkumar_singh · ‎07-17-2017

@wbu no, to achieve HA on the same protocol, you need to register similar protocol hiveserver2 with zookeeper.

rajkumar_singh · ‎07-16-2017

@heta desai yes you can, query hive using hive-jdbc driver( sample - https://github.com/rajkrrsingh/HiveServer2JDBCSample) the same way what you do with normal RDBMS, get the resultset and display using javascript/jquery on the presentation layer.

rajkumar_singh · ‎07-16-2017

@wbu looking at the source code https://github.com/apache/hive/blob/master/jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java#L141 https://github.com/apache/hive/blob/master/jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java#L145 if you register 4 hiveserver2 instances (2 with binary while 2 with http mode) client can connect to any of this one hiveserver2 running with any protocol. so its not a HA between two similar protocol

rajkumar_singh · ‎07-13-2017

@Anurag Mishra you need to run following command, it will return json output,you need to parse json output and get the required details curl --compressed -X GET http://hdp26:8088/ws/v1/cluster/apps?state=RUNNING {"apps":{"app":[{"id":"application_1499066924486_0010","user":"hive","name":"HIVE-971ad1b1-b8c8-425a-a0df-ad50f6cb4798","queue":"llap","state":"RUNNING","finalStatus":"UNDEFINED","progress":0.0,"trackingUI":"ApplicationMaster","trackingUrl":"http://hdp26.hdp.local:8088/proxy/application_1499066924486_0010/","diagnostics":"","clusterId":1499066924486,"applicationType":"TEZ","applicationTags":"","priority":0,"startedTime":1499750778877,"finishedTime":0,"elapsedTime":190496298,"amContainerLogs":"http://hdp26.hdp.local:8042/node/containerlogs/container_e05_1499066924486_0010_01_000001/hive","amHostHttpAddress":"hdp26.hdp.local:8042","allocatedMB":1024,"allocatedVCores":1,"runningContainers":1,"memorySeconds":195067899,"vcoreSeconds":190495,"queueUsagePercentage":16.666668,"clusterUsagePercentage":10.0,"preemptedResourceMB":0,"preemptedResourceVCores":0,"numNonAMContainerPreempted":0,"numAMContainerPreempted":0,"resourceRequests":[{"capability":{"memory":1024,"memorySize":1024,"virtualCores":1},"nodeLabelExpression":"","numContainers":0,"priority":{"priority":0},"relaxLocality":true,"resourceName":"*"}],"logAggregationStatus":"NOT_START","unmanagedApplication":false,"amNodeLabelExpression":""},{"id":"application_1499066924486_0009","user":"hive","name":"llap0","queue":"llap","state":"RUNNING","finalStatus":"UNDEFINED","progress":100.0,"trackingUI":"ApplicationMaster","trackingUrl":"http://hdp26.hdp.local:8088/proxy/application_1499066924486_0009/","diagnostics":"","clusterId":1499066924486,"applicationType":"org-apache-slider","applicationTags":"name: llap,version: 11jul2017,description: llap is a daemon service that works with a cache and works on sql constructs.","priority":1,"startedTime":1499750705675,"finishedTime":0,"elapsedTime":190569500,"amContainerLogs":"http://hdp26.hdp.local:8042/node/containerlogs/container_e05_1499066924486_0009_01_000001/hive","amHostHttpAddress":"hdp26.hdp.local:8042","allocatedMB":4096,"allocatedVCores":2,"runningContainers":2,"memorySeconds":780521363,"vcoreSeconds":381124,"queueUsagePercentage":66.66667,"clusterUsagePercentage":40.0,"preemptedResourceMB":0,"preemptedResourceVCores":0,"numNonAMContainerPreempted":0,"numAMContainerPreempted":0,"resourceRequests":[{"capability":{"memory":512,"memorySize":512,"virtualCores":1},"nodeLabelExpression":"","numContainers":0,"priority":{"priority":0},"relaxLocality":true,"resourceName":"*"},{"capability":{"memory":3584,"memorySize":3584,"virtualCores":1},"nodeLabelExpression":"","numContainers":0,"priority":{"priority":1},"relaxLocality":true,"resourceName":"hdp26.hdp.local"},{"capability":{"memory":3584,"memorySize":3584,"virtualCores":1},"nodeLabelExpression":"","numContainers":0,"priority":{"priority":1},"relaxLocality":false,"resourceName":"/default-rack"},{"capability":{"memory":3584,"memorySize":3584,"virtualCores":1},"nodeLabelExpression":"","numContainers":0,"priority":{"priority":1},"relaxLocality":false,"resourceName":"*"}],"logAggregationStatus":"RUNNING","unmanagedApplication":false,"amNodeLabelExpression":""}]}}

rajkumar_singh · ‎07-13-2017

No this will fetch you RUNNING application from single cluster where you execute it, if you want to monitor remote cluster for RUNNING job you can use YARN RM rest API curl --compressed -X GET http://<cluster1_RM_HOST>:8088/ws/v1/cluster/apps?state=RUNNING curl --compressed -X GET http://<cluster2_RM_HOST>:8088/ws/v1/cluster/apps?state=RUNNING

rajkumar_singh · ‎07-13-2017

try this on yarn cluster, it will fetch you all running application on cluster yarn application -list -appStates RUNNING

rajkumar_singh · ‎07-13-2017

did you miss anything in stacktrace, do you have any error/exception around this exception stating why Initializer fail?

rajkumar_singh · ‎07-11-2017

np @Tomomichi Hirano feel free to accept best answer in this discussion thread so that other user can get benefit from it.

rajkumar_singh · ‎07-10-2017

Did I understand you right: is the NodeManager not used if there is no DataNode installed on the same host? this is not right understanding, NM will be used whether DN is on same host or not, as your installation has slave node where DN is running, if you install NM on these nodes then yarn will only spin container on these nodes, not the master node.

rajkumar_singh · ‎07-10-2017

you can uninstall nodemanager from master node and colocate it with the datanodes, this way you can stop yarn to spin container on master node and since it is running on similar slave node where datanode is running so you need not to worry about the data locality.

rajkumar_singh · ‎07-10-2017

@Tomomichi Hirano ya you can initially go with enabling ranger at hiveserver2 level for now,hiveserver2 will expect that you have read/write permission for hive user on file/folder on hdfs, Additionally, if you install ranger plugin for HDFS then it will be good for you so that you need not mess with hdfs acls and with ranger you can manage at a single place.

Online	Offline
Last Visited	‎08-23-2021 03:30 PM

Date Registered	‎04-25-2016 07:57 AM
Last Visited	‎08-23-2021 03:30 PM
Total Messages Posted	579
Total Kudos Received	608

Re: Why Hive Compaction is failing ?

Re: how to setup queue name for squirrel

Re: How to set the logging level of the hiveserver...

Re: Hive Tez Client Memory

Re: Resource Manager API ?

Re: Is it possible to create materialized view fro...

Re: Why Hive Compaction is failing ?

Re: Is it possible to create materialized view fro...

Re: select count query taking more time

Re: how to setup queue name for squirrel

Creating custom udf and adding udf jar to Hive LLA...

Re: Rest api to get host address in which particul...

Re: LLAP JDBC connection failed

Re: How to set the logging level of the hiveserver...

Re: will hive alter partition impact the query bei...

Re: will hive alter partition impact the query bei...

Re: Hive Tez Client Memory

Re: HDP 2.6 HIVE meta-store failed

Re: Hive command line hung

Re: HDP 2.6 HIVE meta-store failed

Re: HDP 2.6 HIVE meta-store failed

Re: HDP 2.6 HIVE meta-store failed

Re: Hive command line hung

Re: HDP 2.6 HIVE meta-store failed

Re: HiveServer2 Binary and HTTP HA

Re: can we display habse/hive data on html/jquery ...

Re: HiveServer2 Binary and HTTP HA

Re: Resource Manager API ?

Re: script to get top 3 yarn applications in RUNN...

Re: script to get top 3 yarn applications in RUNN...

Re: error in storm

Re: Can we share a set of LLAP daemons with multip...

Re: Avoid Resource Allocation by YARN

Re: Avoid Resource Allocation by YARN

Re: Can we share a set of LLAP daemons with multip...