Member since 
    
	
		
		
		09-25-2015
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                101
            
            
                Posts
            
        
                51
            
            
                Kudos Received
            
        
                25
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 1487 | 08-25-2017 01:06 PM | |
| 1894 | 05-16-2017 12:53 PM | |
| 1624 | 05-15-2017 05:07 PM | |
| 1584 | 05-11-2017 04:19 PM | |
| 1194 | 05-05-2017 12:29 AM | 
			
    
	
		
		
		10-19-2015
	
		
		10:10 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Sorry, my question had a 2 part. Sorry I did not make that clear.   Setting up different kdcs as suggested above works for configuration and HA.   However, the user wants to configure a kdc with a load balancer to ease up the traffic.   if 2 kdcs are setup, then users can still connect to the heavy used one instead of being redirected to the idle one via loadbalancer. Does this make sense?  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-19-2015
	
		
		06:06 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Thanks for the reply all. This was a non issue.   Ambari is reading JMX and JMX is incorrectly aggregating GC Times.   Similar to: https://issues.apache.org/jira/browse/AMBARI-3178  Thanks for the help  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-15-2015
	
		
		09:48 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 [root@hadm0301 ~]# free -c 5    total  used  free  shared  buffers  cached  Mem:  148308580  121816216  26492364  0  24726180  75655352  -/+ buffers/cache:  21434684  126873896  Swap:  2097144  0  2097144 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-15-2015
	
		
		08:14 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Yes but it varies heavily depending the day/time. At times, it could be intensive. I am not sure it is heavy at this moment but let me get that 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-15-2015
	
		
		07:49 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Seeing PSYoungGen GC on YARN/RM nodes aggressively every 8-10 secs:   6171.971: [GC [PSYoungGen: 136875K->512K(144896K)] 252995K->116647K(1542656K), 0.0070630 
secs] [Times: user=0.05 sys=0.00, real=0.01 secs] 
6180.179: [GC [PSYoungGen: 115578K->480K(142336K)] 231714K->116703K(1540096K), 0.0065030 
secs] [Times: user=0.04 sys=0.01, real=0.00 secs] 
6190.316: [GC [PSYoungGen: 141974K->736K(142336K)] 258198K->116975K(1540096K), 0.0059270 
secs] [Times: user=0.03 sys=0.01, real=0.01 secs] 
6200.145: [GC [PSYoungGen: 137784K->672K(140800K)] 254024K->116935K(1538560K), 0.0059910 
secs] [Times: user=0.04 sys=0.00, real=0.01 secs] 
6209.983: [GC [PSYoungGen: 138483K->704K(139776K)] 254746K->117079K(1537536K), 0.0060980 
secs] [Times: user=0.04 sys=0.00, real=0.01 secs] 
6220.037: [GC [PSYoungGen: 137973K->544K(138752K)] 254349K->117157K(1536512K), 0.0036380 
secs] [Times: user=0.02 sys=0.00, real=0.00 secs] 
6228.739: [GC [PSYoungGen: 117752K->768K(137728K)] 234366K->117437K(1535488K), 0.0061930 
secs] [Times: user=0.03 sys=0.02, real=0.00 secs] 
6237.006: [GC [PSYoungGen: 113053K->768K(136704K)] 229723K->117493K(1534464K), 0.0037210 
secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
6244.956: [GC [PSYoungGen: 107217K->672K(135680K)] 223942K->117589K(1533440K), 0.0080520 
secs] [Times: user=0.03 sys=0.03, real=0.01 secs] 
6255.009: [GC [PSYoungGen: 133601K->512K(134656K)] 250518K->117517K(1532416K), 0.0070160 
secs] [Times: user=0.04 sys=0.00, real=0.01 secs] 
6265.007: [GC [PSYoungGen: 131612K->672K(134144K)] 248618K->117821K(1531904K), 0.0039770 
secs] [Times: user=0.04 sys=0.00, real=0.00 secs] 
6274.880: [GC [PSYoungGen: 129679K->815K(132608K)] 246828K->118132K(1530368K), 0.0036740 
secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 
6284.782: [GC [PSYoungGen: 129427K->608K(132096K)] 246744K->118164K(1529856K), 0.0039850 
secs] [Times: user=0.04 sys=0.00, real=0.01 secs] 
6294.809: [GC [PSYoungGen: 129803K->576K(130560K)] 247360K->118260K(1528320K), 0.0037790 
secs] [Times: user=0.03 sys=0.00, real=0.00 secs] 
6297.736: [GC [PSYoungGen: 39607K->544K(129536K)] 157291K->118316K(1527296K), 0.0035240 secs] 
[Times: user=0.02 sys=0.00, real=0.00 secs]   Is this normal?   Is there a way to properly tweak it?   Looks like the new size is set to 144M, should this be increased?   Thank you !  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache YARN
			
    
	
		
		
		10-14-2015
	
		
		01:29 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 While trying to configure kdc with a Virtual IP, we are getting the below error   Could not find user in Kerberos database  Ambari fails the install.   VIP is needed for loadbalancing and HA.   When single node IP is used for kdc, then the install is successful.  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache Ambari
			
    
	
		
		
		10-08-2015
	
		
		03:37 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Yes and it gave more insight into what was wrong:   FAILED: SemanticException [Error 10007]: Ambiguous column reference visit_num in features_and_label  Customer had a duped column.  Thank you for the tip.  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-07-2015
	
		
		07:18 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hive v13,hitting the semantic error:   2015-10-07 05:12:40,988 ERROR ql.Driver (SessionState.java:printError(547)) - FAILED: NullPointerException null
java.lang.NullPointerException
at org.apache.hadoop.hive.ql.optimizer.ColumnPrunerProcFactory$ColumnPrunerSelectProc.process(ColumnPrunerProcFactory.java:618)
at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
at org.apache.hadoop.hive.ql.optimizer.ColumnPruner$ColumnPrunerWalker.walk(ColumnPruner.java:166)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
at org.apache.hadoop.hive.ql.optimizer.ColumnPruner.transform(ColumnPruner.java:129)
at org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:146)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9414)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:427)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:323)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:980)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1045)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:916)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:359)
at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:456)
at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:466)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:749)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)  Looks very similar to:   https://issues.apache.org/jira/browse/HIVE-9185  Looks like the ability to disable pruner was taken away on Hive13:  hive.optimize.cp=false https://issues.apache.org/jira/browse/HIVE-4113  Is there another way to disable the pruner temporarily?  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache Hive
- 
						
							
		
			Apache Tez
			
    
	
		
		
		10-05-2015
	
		
		04:52 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Has anyone seen an intermittent error while exporting from HDP to external DB via Teradata Connector:   2015-09-22 13:03:29,969 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : com.teradata.connector.common.exception.ConnectorException: java.net.ConnectException: Connection refused 
at java.net.PlainSocketImpl.socketConnect(Native Method) 
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) 
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) 
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) 
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) 
at java.net.Socket.connect(Socket.java:579) 
at java.net.Socket.connect(Socket.java:528) 
at java.net.Socket.<init>(Socket.java:425) 
at java.net.Socket.<init>(Socket.java:208) 
at com.teradata.connector.teradata.TeradataInternalFastloadOutputFormat.getRecordWriter(TeradataInternalFastloadOutputFormat.java:301) 
at com.teradata.connector.common.ConnectorOutputFormat$ConnectorFileRecordWriter.<init>(ConnectorOutputFormat.java:84) 
at com.teradata.connector.common.ConnectorOutputFormat.getRecordWriter(ConnectorOutputFormat.java:33) 
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:644) 
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) 
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) 
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) 
at java.security.AccessController.doPrivileged(Native Method) 
at javax.security.auth.Subject.doAs(Subject.java:415) 
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) 
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)  The connection string is:   hadoop jar /usr/hdp/2.2.4.5-1/sqoop/lib/teradata-connector-1.3.5-hadoop220.jar com.teradata.hadoop.tool.TeradataExportTool -url jdbc:teradata://<server>/Database=PR_ODMM_STG -username PR_BATCH_ODMM_SQL -password xxxxx -errortabledatabase PR_ODMM_STG -jobtype hdfs -sourcepaths /user/hive/warehouse/edw_odmm_stg.db/omn_aggregates/batch_id=15/agg_tab_partition=STG_INTERNAL_SEARCH_KEYWORD_1 -nummappers 5 -separator \u001E -targettable STG_INTERNAL_SEARCH_KEYWORD_1 -errortablename NTERNAL_SEARCH_KEYWORD_1 -method  internal.fastload  & 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
- 
						
							
		
			Apache YARN
			
    
	
		
		
		09-30-2015
	
		
		05:06 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Good find. Thanks  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		- « Previous
- Next »
 
         
					
				













