Member since 
    
	
		
		
		09-25-2015
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                82
            
            
                Posts
            
        
                93
            
            
                Kudos Received
            
        
                17
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 4614 | 06-06-2017 09:57 AM | |
| 1367 | 03-01-2017 10:26 AM | |
| 1465 | 11-22-2016 10:32 AM | |
| 1146 | 08-09-2016 12:05 PM | |
| 2141 | 08-08-2016 03:57 PM | 
			
    
	
		
		
		03-30-2016
	
		
		03:06 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 have you verified that the sqoop command works by itself? run it manually on the command line outside of oozie.  are you running your workflow through hue or via the oozie command line? if through hue, try running it on the oozie command line to verify it works as well. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		03-30-2016
	
		
		03:02 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		2 Kudos
		
	
				
		
	
		
					
							 Hi @nejm hadj does NiFi write the files to one directory? You should be able to specify an EXTERNAL table with the location as a directory. Then all the files in the dir will be part of the one table.  For example:  CREATE EXTERNAL TABLE tweets (col1 string, col2 string)
LOCATION '/path/to/dir';  Obviously use a JSON SerDe to parse the data into your table columns as well, but the LOCATION clause is sufficient to load all your files into your table. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-20-2016
	
		
		11:08 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 @Neeraj Sabharwal I don't have a doc or demo, but it's simple enough to demonstrate. Enable SBA (should be already by default) and and then give the hive warehouse dir e.g. 750 permissions for hive:hadoop (recursively). Then if people try to access tables through Hive CLI as any other user, they will get a permission denied error. This won't apply to access via HiveServer2 because you'd manage table and DB permissions via Ranger Hive repository or SQL Standard authorisation. @Alex Miller's suggestion goes on top of that as well. To be truly secure, users must come in via Knox. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-20-2016
	
		
		12:09 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		5 Kudos
		
	
				
		
	
		
					
							 I think you have to make sure Storage Based Authorization (SBA) is enabled for the Metastore   
 hive.metastore.pre.event.listeners = org.apache.hadoop.hive.ql.security.authorization.AuthorizationPreEventListener  hive.security.metastore.authorization.manager = org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider    And then lock down filesystem permissions for data warehouse dir. I don't think there's a way to actually prevent users from issuing the "hive" command. I may be wrong, though - happy to be corrected. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-15-2016
	
		
		09:38 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		3 Kudos
		
	
				
		
	
		
					
							 Hi @Junichi Oda - This is expected behaviour and it is the reason why it is recommended to have all hive processes run as hive user when you secure Hive with ranger.  There are two options in order to secure access to hive with Ranger :  Solution 1  Use both a repository HDFS and Hive to handle rights   
Keep "run as end user instead of hive" (hive.server2.enable.doAs=true)   This means the dual maintenance that you describe  Solution 2  Give rights to the hive user on the /apps/hive/warehouse arborescence in Ranger HDFS repository   Lock down filesystem permissions on HDFS (for example, chmod 750)   Use the Ranger Hive repository to handle rights on Hive tables   Run as hive instead of end user (hive.server2.enable.doAs=false)  ---  Solution 2 is the way to go. You may be concerned about auditability, but the Hive audits in Ranger will show the correct user. The HDFS audits and the YARN audits will still show "hive" yes, but you will be able to tell who ran the query. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-24-2015
	
		
		12:48 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 Did you attempt and abort an upgrade already?   Can you please check the clusterconfigmapping table in the Ambari DB and make sure there are no duplicates in the type_name column?   If there are, and then make sure that only 1 of them has a value of "1" in the "selected" column - the one with a later created_timestamp value. Take a backup of the Ambari DB (!!) and then update the "selected" value to 0 for the other one. Then re-attempt the upgrade. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-22-2015
	
		
		06:06 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		2 Kudos
		
	
				
		
	
		
					
							 EDIT: oops - just read the end bit of your post... still try to do it in the shell action itself and see?  ---  I believe this is a known limitation in non-secure clusters, whereby the containers are running as YARN user and not running as user xyz.   Try to pass in the HADOOP_USER_NAME variable as illustrated below:   $ `export HADOOP_USER_NAME=xyz ; hive -hiveconf hive.execution.engine=mr -e 'show databases' ` 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-22-2015
	
		
		05:56 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		5 Kudos
		
	
				
		
	
		
					
							 According to this Hive Language documentation, you can achieve the same thing using regular expressions:  A SELECT statement can take regex-based column specification in Hive releases prior to 0.13.0, or in 0.13.0 and later releases if the configuration property hive.support.quoted.identifiers is set to none.   * We use Java regex syntax. Try http://www.fileformat.info/tool/regex.htm for testing purposes.   * The following query selects all columns except ds and hr.   SELECT `(ds|hr)?+.+` FROM sales 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-10-2015
	
		
		05:47 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi @Bhupendra Mishra  You don't need the --table argument - it's only relevant for imports and exports (not eval).   Try this:  sqoop eval --connect jdbc:mysql://localhost.localdomain/sqoop_test --username root --query "SELECT * FROM emp" 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-10-2015
	
		
		12:10 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Do you have Kerberos enabled on this cluster? Also - are you using HDP 2.3.0 or HDP 2.3.2? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		 
         
					
				













