Member since 
    
	
		
		
		09-13-2015
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                59
            
            
                Posts
            
        
                18
            
            
                Kudos Received
            
        
                6
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 3255 | 12-19-2017 08:14 PM | |
| 2818 | 10-11-2017 02:21 PM | |
| 3692 | 06-12-2017 09:26 PM | |
| 5633 | 06-08-2017 01:36 PM | |
| 2143 | 11-04-2016 08:35 PM | 
			
    
	
		
		
		06-08-2017
	
		
		01:36 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 It appears that this is expected behavior.    I discovered a jira pertaining to it here: https://issues.apache.org/jira/browse/HIVE-13865 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-07-2017
	
		
		07:18 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I have noticed an issue in Hive 1.2 where char data is rtrimmed when converted to string.  This doesn't occur when converting varchar to string.  Is this a bug? or a feature?  select length(cast('abc    ' as char(10))), 
	   length(cast('abc    ' as varchar(10))),
	   length(cast('abc    ' as string));
_c0,_c1,_c2
3,7,7
  In the above example I would expect the length of the char field to be 10. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache Hive
 
			
    
	
		
		
		06-06-2017
	
		
		07:41 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 The individual file sizes is not stored in the metastore, so there is not a way to query them directly  From within the hive shell you can execute HDFS commands such as  dfs -ls /path/to/table   to see the individual files and their sizes.    If you're interested in the total data size of the table, you can execute:   DESCRIBE FORMATTED table_name;    and look for the table parameter named totalSize. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		05-24-2017
	
		
		03:55 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 The hive table column names are stored in the metastore.  Assuming you are using an external hive table on the txt file(s) you will need to either update the DDL for the table, or drop and re-create the table with the correct columns.  Dropping an external table does not delete the underlying data. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		04-20-2017
	
		
		08:36 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 It includes Spark 1.6 and 2.1.  Hbase 1.1.2  https://2xbbhjxc6wk3v21p62t8n4d4-wpengine.netdna-ssl.com/wp-content/uploads/2017/03/Asparagus-2.6.png 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-06-2017
	
		
		12:25 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I am having an issue where occasionally a node will drop out of the cluster and will not rejoin after restarting.  I am running NiFi 1.1.0 on VMs.  3 node HDF cluster, with 2 of the nodes running NiFi and the third running ambari/ranger/LogSearch/.  On the running node it will display 1/2 nodes connected.  On the failed node the UI stays at the "voting on a flow" screen.  The only way I can get the node to reconnect to the cluster is by removing the flowfile_repository directory.  Unfortunately I can't post logs without significant cleansing, but if there is something in particular I can post. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache NiFi
 
			
    
	
		
		
		11-04-2016
	
		
		08:38 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 I had the same issue.  The root cause of my issue was the hive user account (that the compactor was running under) did not have write permissions to the directory that the hive database was in.  After adjusting the permissions I was able to run the compactor succesfully.  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		11-04-2016
	
		
		08:35 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Discovered the problem was the hive user account did not have write permissions to the directory containing the hive database.  After fixing the permissions the compactor ran succesfully. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		11-03-2016
	
		
		02:02 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 If the date is an attribute named 'dt' you can use an attribute variable in the directory path using the syntax ${dt} . 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		11-03-2016
	
		
		01:44 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 In my lab environment I have a transactional hive table setup that is receiving data from NiFi.  Everything works except the number of delta directories is blowing up because the compactor is not successfully running.  It appears to be looking for a _tmp file that does not exist.  This is the error that I am getting:  No of maps and reduces are 0 job_1478131263487_0009
Job commit failed: java.io.FileNotFoundException: File hdfs://node2.localdomain:8020/apps/hive/warehouse/stock.db/hdp/_tmp_80e50691-c9fe-485a-8e1e-ba20331c7d97 does not exist.
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:904)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:113)
at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:966)
at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:962)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:962)
at org.apache.hadoop.hive.ql.txn.compactor.CompactorMR$CompactorOutputCommitter.commitJob(CompactorMR.java:776)
at org.apache.hadoop.mapred.OutputCommitter.commitJob(OutputCommitter.java:291)
at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:285)
at org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:237)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
		
			
				
						
							Labels:
						
						
		
			
	
					
			
		
	
	
	
	
				
		
	
	
- Labels:
 - 
						
							
		
			Apache Hive
 - 
						
							
		
			Apache NiFi
 
- « Previous
 - Next »