Member since 
    
	
		
		
		06-28-2016
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                55
            
            
                Posts
            
        
                3
            
            
                Kudos Received
            
        
                1
            
            
                Solution
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 1239 | 07-22-2016 02:08 PM | 
			
    
	
		
		
		09-16-2019
	
		
		09:21 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @tmater Thank you for the update.  What if some one alters the table, without knowing this fact that it would not work in impala.  If we have a huge data set(historical data) that the user will be unable to view in impala.  what is best way to overcome this.  1.Is there a way we can edit the parquet files metadata in impala either thru spark/from command line ?.  2.we tried creating a temporary table with new schema and copying data from old table with Lateral view explode(on array/complex data type) in hive(manually with hive queries) and it is taking lot of our cluster resource and terminating without doing anything.  3.Even we tried to split the historical data(source) with date range and it is not helping us.     I believe Lateral view explode is taking lot of resource on a historical data and causing to terminate.     Notes: we have complex data type in our source table and so we are using Lateral view explode in hive     Any work around either in spark/hive/impala anything that will help to deal with historical and current data in impala would be really helpful.     Thank you for your help on this 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		09-16-2019
	
		
		01:31 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 1.We have hive table with below schema. 
 create external table precision_1  (  id String,  prec decimal(7,2)  )  partitioned by (date string)  stored as parquet; 
 2.insert into precision_1 partition(date='2019-02-01')  values ('1', 2.34); 
 3.insert into precision_1 partition(date='2019-02-01')  values ('2', 3.45); 
 3. 'SELECT * from precision_1' works in both impala and Hive 
 4.We did following in Hive and invalidated metadata in impala: 
 ALTER TABLE precision_1 CHANGE prec prec decimal(9,6); 
 insert into precision_1 partition(date='2019-01-02')  values ('1', 3.765894); 
 'SELECT * from precision_1' works only in hive but not in imala 
 In impala throws the below erorr. 
 Bad status for request TFetchResultsReq(fetchType=0, operationHandle=TOperationHandle(hasResultSet=True, modifiedRowCount=None, operationType=0, operationId=THandleIdentifier(secret='pq)"\xf9LNN\x00\x00\x00\x00\xb1Z\x17{', guid='pq)"\xf9LNN\x00\x00\x00\x00\xb1Z\x17{')), orientation=4, maxRows=100): TFetchResultsResp(status=TStatus(errorCode=None, errorMessage="File 'hdfs://devana1/user/hive/warehouse/precision_1/date=2019-02-01/000000_0_copy_1' column 'prec' has a scale that does not match the table metadata scale. File metadata scale: 2 Table metadata scale: 6\n", sqlState='HY000', infoMessages=None, statusCode=3), results=None, hasMoreRows=None) 
   
 kindly suggest 
   
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache Hive
 - 
						
							
		
			Apache Impala
 
			
    
	
		
		
		07-27-2016
	
		
		03:27 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @slachterman Understood, Thanks for the detailed explaination!!!  What if i have to re import again and output directory already exist, is there options for sqoop import with overwrite option? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-27-2016
	
		
		02:34 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @slachterman Thanks it worked now!!! where can i find that data in HDFS if no path is specified ?  So It has to be changed if i am importing from oracle/postgresql ? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-27-2016
	
		
		01:34 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi @slachterman   when i am trying to run the sqoop import command i am getting the below error.      Thanks,  Ravikumar. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache Sqoop
 
			
    
	
		
		
		07-27-2016
	
		
		07:13 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Sindhu  I tried using sqoop import command and it gives me the below error.     
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-27-2016
	
		
		06:51 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Michael Young I tried even using lowercase p it couldn't connect me.  But I just tried with ssh root@192.168.228.128 it works.  Thank you 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-26-2016
	
		
		03:09 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Michael Young Do you mean i should use cygwin/putty to connect to sandbox and then try running these commands so that i can scroll thru my result log?  I was trying to connect to sandbox using cygwin with below command and it does not recognize the command         
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		07-26-2016
	
		
		02:48 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi All,  When I execute any command in sandbox, if my command has any syntax error/if something is going wrong while execution, it gives me whole list of error and it takes me to bottom of error how can i see the full error in sandbox from the begining.  My intention is scroll thru the error and see each and every line of my error.     
						
					
					... View more