Member since 
    
	
		
		
		06-08-2014
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                33
            
            
                Posts
            
        
                6
            
            
                Kudos Received
            
        
                1
            
            
                Solution
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 2790 | 11-23-2018 11:18 AM | 
			
    
	
		
		
		12-07-2018
	
		
		01:15 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I have enabled ACID transactions in Hive-3 within HDP-3 at cluster level. When i am trying to disable it on session level using below properties and creating new managed table using this (TBLPROPERTIES ('orc.compress'='ZLIB'),('transactional'='true')) table properties. It's giving below error :  set hive.optimize.index.filter=false;   set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager ;   set hive.compactor.initiator.on=false;   set hive.compactor.worker.threads=0;   set hive.strict.managed.tables=false;   INFO  : Starting task [Stage-0:DDL] in serial mode
ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Table ref_edw4x_qn1useh1.dummy failed strict managed table checks due to the following reason: Table is marked as a managed table but is not transactional.)
INFO  : Completed executing command(queryId=hive_20181207104022_6a3478f0-9b8a-44db-a9a4-4d2ab5fe2b11); Time taken: 0.076 seconds
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Table ref_edw4x_qn1useh1.dummy failed strict managed table checks due to the following reason: Table is marked as a managed table but is not transactional.) (state=08S01,code=1) 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache Hive
			
    
	
		
		
		11-23-2018
	
		
		11:18 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Prabhu M @Mahesh Balakrishnan : Issue has been resolved.  It's a bug with Tez version 0.9.0 (HDP-3.0.0.0) which they have fixed in Tez 0.9.1 version(HDP-3.0.1.0)     
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		11-12-2018
	
		
		06:07 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 18/11/12 10:37:25 [main]: INFO jdbc.HiveConnection: Connected to server1.test.com:10000
Connected to: Apache Hive (version 3.0.0.3.0.0.0-1334)
Driver: Hive JDBC (version 3.0.0.3.0.0.0-1334)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 3.0.0.3.0.0.0-1334 by Apache Hive
0: jdbc:hive2://server1.test.com:> select count(*) from lnd_edw4x_qn1useh1.redemption_fact_t;   INFO  : Compiling command(queryId=hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216): select count(*) from lnd_edw4x_qn1useh1.redemption_fact_t   INFO  : Semantic Analysis Completed   INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216); Time taken: 1.218 seconds   INFO  : Executing command(queryId=hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216): select count(*) from lnd_edw4x_qn1useh1.redemption_fact_t   INFO  : Query ID = hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216
INFO  : Total jobs = 1
INFO  : Launching Job 1 out of 1
INFO  : Starting task [Stage-1:MAPRED] in serial mode   INFO  : Subscribed to counters: [] for queryId: hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216   INFO  : Tez session hasn't been created yet. Opening session   INFO  : Dag name: select count(*) from ...h1.redemption_fact_t (Stage-1)   ERROR : Status: Failed
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1542008498326_0022_1_00, diagnostics=[Vertex vertex_1542008498326_0022_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: redemption_fact_t initializer failed, vertex=vertex_1542008498326_0022_1_00 [Map 1], java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.adl.AdlFileSystem not found           at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2595)           at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3320)  I am already having below jars in Hadoop/hive libs still it's not working:  hadoop-azure-3.0.0.3.0.0.0-1334.jar, hadoop-azure-datalake-3.0.0.3.0.0.0-1334.jar, azure-data-lake-store-sdk-2.2.7.jar 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache Hive
			
    
	
		
		
		07-06-2018
	
		
		01:50 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I am having only hiveserver2 service on Server2.     
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-06-2018
	
		
		12:47 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I have configured Hiverserver2 in HA mode and started in INFO mode and it is generating logs in INFO mode on server1 but on server2 it's generating in DEBUG mode. I also checked in configuration there is no different configuration for both hiveservers.  Below are running hiveserver2 processes on server1 & server2.  Server1:~> ps -ef | grep hiveserver2  
hive   6717  1  2 Jul02 ?        02:02:18 /usr/lib/jvm/jdk1.8.0_71/bin/java -Xmx1024m -Dhdp.version=2.5.3.0-37 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.5.3.0-37 -Dhadoop.log.dir=/hadoop/var/log/hadoop/hive -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.5.3.0-37/hadoop -Dhadoop.id.str=hive -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.5.3.0-37/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx1024m -Xmx12065m -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=/usr/hdp/2.5.3.0-37/hive/bin/../conf/parquet-logging.properties -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.5.3.0-37/hive/lib/hive-service-1.2.1000.2.5.3.0-37.jar org.apache.hive.service.server.HiveServer2 --hiveconf hive.aux.jars.path=file:///usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar -hiveconf hive.metastore.uris=  -hiveconf hive.log.file=hiveserver2.log -hiveconf hive.log.dir=/hadoop/var/log/hive   =======================================  Server2:~> ps -ef | grep hiveserver2   hive       4459  1  3  Jul02 ?  03:30:22  /usr/lib/jvm/jdk1.8.0_71/bin/java -Xmx1024m -Dhdp.version=2.5.3.0-37 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.5.3.0-37 -Dhadoop.log.dir=/hadoop/var/log/hadoop/hive -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.5.3.0-37/hadoop -Dhadoop.id.str=hive -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.5.3.0-37/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx1024m -Xmx12065m -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=/usr/hdp/2.5.3.0-37/hive/bin/../conf/parquet-logging.properties -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.5.3.0-37/hive/lib/hive-service-1.2.1000.2.5.3.0-37.jar org.apache.hive.service.server.HiveServer2 --hiveconf hive.aux.jars.path=file:///usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar -hiveconf hive.metastore.uris=  -hiveconf hive.log.file=hiveserver2.log -hiveconf hive.log.dir=/hadoop/var/log/hive    Please respond. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache Hive
			
    
	
		
		
		08-17-2017
	
		
		06:25 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hey Nitin,  Changed the configs like,  object class=user  Distinguished name attribute=distinguishedName  Group member attribute=member  And re-synced users. It is working now.  Thanks for help. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-17-2017
	
		
		06:22 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Yes this helps a lot. Thanks for quick answer. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-17-2017
	
		
		06:08 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I have below set of properties,  authentication.ldap.groupMembershipAttr=memberOf   authentication.ldap.groupNamingAttr=cn   authentication.ldap.groupObjectClass=group   authentication.ldap.userObjectClass=person   authentication.ldap.usernameAttribute=sAMAccountName  authentication.ldap.dnAttribute=dn  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-17-2017
	
		
		04:52 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Is there a way to find out - List Users/jobs exhausting the YARN queues , running with more than 4 TB of memory and 400 containers in last 2 weeks 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache YARN
 
        













