Member since 
    
	
		
		
		04-25-2016
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                579
            
            
                Posts
            
        
                609
            
            
                Kudos Received
            
        
                111
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 2925 | 02-12-2020 03:17 PM | |
| 2136 | 08-10-2017 09:42 AM | |
| 12471 | 07-28-2017 03:57 AM | |
| 3410 | 07-19-2017 02:43 AM | |
| 2522 | 07-13-2017 11:42 AM | 
			
    
	
		
		
		02-14-2020
	
		
		09:02 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Ellyly  the NPE you got is side effect of Exception you have in stage-3/4/5, can you please upload the Error stack you got before the NPE, I am assuming that you are running hive on tez?  I think ReExecOverlayPlugin failing with NPE and shallowing the exception at client-side (fix for this has been identified as Hive-19204 but you should be able to see the real cause in HS2 logs  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-12-2020
	
		
		03:17 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I think error log is pretty informative, getGroups call for $Superuser does not get any group associated with the user, you might need to run "hdfs groups" command on the HMS node for this user if that the case and fix the env issue. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-12-2020
	
		
		12:50 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Ellyly  can you please post the complete stack trace from the HiveServer2 logs? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-14-2017
	
		
		09:45 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 if you are running query on tez then this will be useful to follow https://community.hortonworks.com/articles/22419/hive-on-tez-performance-tuning-determining-reducer.html 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-10-2017
	
		
		09:42 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 add following in connection string   tez.queue.name=<queue_name> 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-02-2017
	
		
		08:15 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		7 Kudos
		
	
				
		
	
		
					
							 Creating and running Temporary functions are discouraged while running a query on LLAP because of security reason since many users are sharing same instances of LLAP, it can create a conflict but still, you can create temp functions using add jar and hive.llap.execution.mode=auto.   with exclusive llap execution mode(hive.llap.execution.mode=only) you will run into the ClassNotFoundException, hive.llap.execution.mode=auto will allow some part of query(map tasks) to run in the tez container.   Here are steps to create a custom permanent function in LLAP(steps are tested on HDP-260)   1. create a jar for UDF function (in this case I am using simple udf):  git clone https://github.com/rajkrrsingh/SampleCode 
mvn clean package  2. upload the target/SampleCode.jar to the node where HSI is running(in my case I have copied it to /tmp directory)   3. add jar to hive_aux_jars (goto Ambari--> hive --> config --> hive-interactive-env template)  export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH:/tmp/SampleCode.jar
  4. add the jar to Auxillary JAR list (goto Ambari--> hive --> config --> Auxillary JAR list)  Auxillary JAR list=/tmp/SampleCode.jar  5. restart LLAP   6. create Permanent Custom function  connect to HSI using beeline 
create FUNCTION CustomLength as 'com.rajkrrsingh.hiveudf.CustomLength'; 
describe function CustomLength; 
select CustomLength(description) from sample_07 limit 1;
  7. check where the SampleCode.jar localized  root@hdp26 container_e06_1501140901077_0019_01_000002]# pwd
/hadoop/yarn/local/usercache/hive/appcache/application_1501140901077_0019/container_e06_1501140901077_0019_01_000002
[root@hdp26 container_e06_1501140901077_0019_01_000002]# find . -iname sample*
./app/install/lib/SampleCode.jar 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
	
					
			
		
	
	
	
	
				
		
	
	
			
    
	
		
		
		07-28-2017
	
		
		03:57 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @mungeol heo  you can change the logging level of hiveserver2 using ambari, go to hive--> config --> Advanced hive-log4j --> replace hive.root.logger=INFO,DRFA with hive.root.logger=DEBUG,DRFA and restart hiveserver2 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-19-2017
	
		
		02:43 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Gagan Brahmi modify tez.am.resource.memory.mb in ambari tez configuration, restart tez and hive client. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-13-2017
	
		
		11:42 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							@Anurag Mishra you need to run following command, it will return json output,you need to parse json output and get the required details  curl --compressed -X GET http://hdp26:8088/ws/v1/cluster/apps?state=RUNNING
{"apps":{"app":[{"id":"application_1499066924486_0010","user":"hive","name":"HIVE-971ad1b1-b8c8-425a-a0df-ad50f6cb4798","queue":"llap","state":"RUNNING","finalStatus":"UNDEFINED","progress":0.0,"trackingUI":"ApplicationMaster","trackingUrl":"http://hdp26.hdp.local:8088/proxy/application_1499066924486_0010/","diagnostics":"","clusterId":1499066924486,"applicationType":"TEZ","applicationTags":"","priority":0,"startedTime":1499750778877,"finishedTime":0,"elapsedTime":190496298,"amContainerLogs":"http://hdp26.hdp.local:8042/node/containerlogs/container_e05_1499066924486_0010_01_000001/hive","amHostHttpAddress":"hdp26.hdp.local:8042","allocatedMB":1024,"allocatedVCores":1,"runningContainers":1,"memorySeconds":195067899,"vcoreSeconds":190495,"queueUsagePercentage":16.666668,"clusterUsagePercentage":10.0,"preemptedResourceMB":0,"preemptedResourceVCores":0,"numNonAMContainerPreempted":0,"numAMContainerPreempted":0,"resourceRequests":[{"capability":{"memory":1024,"memorySize":1024,"virtualCores":1},"nodeLabelExpression":"","numContainers":0,"priority":{"priority":0},"relaxLocality":true,"resourceName":"*"}],"logAggregationStatus":"NOT_START","unmanagedApplication":false,"amNodeLabelExpression":""},{"id":"application_1499066924486_0009","user":"hive","name":"llap0","queue":"llap","state":"RUNNING","finalStatus":"UNDEFINED","progress":100.0,"trackingUI":"ApplicationMaster","trackingUrl":"http://hdp26.hdp.local:8088/proxy/application_1499066924486_0009/","diagnostics":"","clusterId":1499066924486,"applicationType":"org-apache-slider","applicationTags":"name: llap,version: 11jul2017,description: llap is a daemon service that works with a cache and works on sql constructs.","priority":1,"startedTime":1499750705675,"finishedTime":0,"elapsedTime":190569500,"amContainerLogs":"http://hdp26.hdp.local:8042/node/containerlogs/container_e05_1499066924486_0009_01_000001/hive","amHostHttpAddress":"hdp26.hdp.local:8042","allocatedMB":4096,"allocatedVCores":2,"runningContainers":2,"memorySeconds":780521363,"vcoreSeconds":381124,"queueUsagePercentage":66.66667,"clusterUsagePercentage":40.0,"preemptedResourceMB":0,"preemptedResourceVCores":0,"numNonAMContainerPreempted":0,"numAMContainerPreempted":0,"resourceRequests":[{"capability":{"memory":512,"memorySize":512,"virtualCores":1},"nodeLabelExpression":"","numContainers":0,"priority":{"priority":0},"relaxLocality":true,"resourceName":"*"},{"capability":{"memory":3584,"memorySize":3584,"virtualCores":1},"nodeLabelExpression":"","numContainers":0,"priority":{"priority":1},"relaxLocality":true,"resourceName":"hdp26.hdp.local"},{"capability":{"memory":3584,"memorySize":3584,"virtualCores":1},"nodeLabelExpression":"","numContainers":0,"priority":{"priority":1},"relaxLocality":false,"resourceName":"/default-rack"},{"capability":{"memory":3584,"memorySize":3584,"virtualCores":1},"nodeLabelExpression":"","numContainers":0,"priority":{"priority":1},"relaxLocality":false,"resourceName":"*"}],"logAggregationStatus":"RUNNING","unmanagedApplication":false,"amNodeLabelExpression":""}]}}
 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-11-2017
	
		
		02:22 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 np @Tomomichi Hirano feel free to accept best answer in this discussion thread so that other user can get benefit from it. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		 
        













