Member since 
    
	
		
		
		09-26-2015
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                48
            
            
                Posts
            
        
                29
            
            
                Kudos Received
            
        
                6
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 7598 | 10-25-2016 12:53 PM | |
| 11030 | 10-22-2016 10:22 PM | |
| 5796 | 10-22-2016 09:34 PM | |
| 6703 | 10-21-2016 09:56 PM | |
| 2788 | 07-17-2016 05:26 PM | 
			
    
	
		
		
		07-14-2016
	
		
		07:28 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Krishna Srinivas   Have you tried the Falcon mirroring feature ? Instead of cluster to cluster replication, you can try replicating to different directories in the same cluster.  http://hortonworks.com/hadoop-tutorial/mirroring-datasets-between-hadoop-clusters-with-apache-falcon/  https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_data_governance/content/section_mirroring_data_falcon.html  https://falcon.apache.org/HDFSDR.html 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-14-2016
	
		
		03:59 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @ANSARI FAHEEM AHMED   Are you referring to Hive/Tez job container sizes ? If yes, you can go to hive CLI and try   set hive.tez.container.size;  or if it a mapreduce job , you can try the same set command for mapreduce mapper or reducer mb.memory properties.  if it is a generic YARN container size for any particular YARN application, then the containers are JVM processes, you can use yarn application commands to get the application attempt id and using the application attempt id, you can list the containers running for that APP. Doing a ps aux | grep <container pid> should give you enough details about the container size. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-14-2016
	
		
		03:43 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @ANSARI FAHEEM AHMED   Are you referring to the running jobs on the cluster, as they acquire resources on YARN, the namenode heap is increasing ? If yes, your processes might be under the hood making lot of namenode requests which might be the reason for increase in namenode heap size as well. The namenode heap size will eventually get down after it hits Garbage Collection. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		05-23-2016
	
		
		02:33 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							@Manoj Dhake Look at the below link for Atlas 0.7  http://atlas.incubator.apache.org/Bridge-Falcon.html  Hope this helps! 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		04-05-2016
	
		
		05:41 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Is there any data encryption option for Spark Thrift Server ? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
	
					
			
		
	
	
	
	
				
		
	
	
			
    
	
		
		
		01-13-2016
	
		
		02:58 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		3 Kudos
		
	
				
		
	
		
					
							 We need to set up a HDP cluster based on Isilon storage and customer is asking how much impact would it have on the CPU usage on Isilon nodes ? Currently the Isilon cluster is shared with other work loads as well. What are our experiences around this ? Would the namenode operations in Isilon cause lot of CPU spikes which results in degradation in performance with other workloads on Isilon ? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache Hive
 
			
    
	
		
		
		11-17-2015
	
		
		07:46 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Trying to import table data from Sybase table to Hive using the below command:   sqoop import --verbose  --driver com.sybase.jdbc4.jdbc.SybDriver --connect jdbc:sybase:Tds:dbgbl-tst:8032/DATABASE=trim_bw --username hrongali -P --table trim_bw..account --hive-database trim_bw --hive-table account --hive-import -m 1  Sqoop is generating the below alias(AS trim_bw..account) which is failing to execute in Sybase and the below exception is thrown:  2015-11-17 14:29:48,511 INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: SELECT col_1, col_2, col_3, col_4 FROM trim_bw..account AS trim_bw..account WHERE ( 1=1 ) AND ( 1=1 )
2015-11-17 14:29:48,514 ERROR [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Top level exception: 
com.sybase.jdbc4.jdbc.SybSQLException: Incorrect syntax near '.'.
	at com.sybase.jdbc4.tds.Tds.processEed(Tds.java:4084)
	at com.sybase.jdbc4.tds.Tds.nextResult(Tds.java:3174)
	at com.sybase.jdbc4.tds.Tds.getResultSetResult(Tds.java:3940)
	at com.sybase.jdbc4.tds.TdsCursor.open(TdsCursor.java:328)
	at com.sybase.jdbc4.jdbc.SybStatement.executeQuery(SybStatement.java:2370)
	at com.sybase.jdbc4.jdbc.SybPreparedStatement.executeQuery(SybPreparedStatement.java:264)
	at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
	at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
  Note: Tried with out giving the database name in the --table parameter, but the table object is not being recognized with that convention 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache Sqoop
 
			
    
	
		
		
		11-12-2015
	
		
		04:00 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		3 Kudos
		
	
				
		
	
		
					
							 Sort Bucket the Hive table and Read the bucketed Hive table in Mapreduce program and hit Hbase when the Key changes. Requires programming effor, but very effective. Bucketing the Hive table will make sure that a particular key goes to only one bucket, so you hit Hbase Once for a particular key. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		11-09-2015
	
		
		04:58 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		2 Kudos
		
	
				
		
	
		
					
							 The below Blog provides very good guideline too:  http://hortonworks.com/blog/best-practices-for-hive-authorization-using-apache-ranger-in-hdp-2-2/ 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		11-07-2015
	
		
		09:35 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Thanks Pardeep ! 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		- « Previous
 - Next »