Member since 
    
	
		
		
		07-30-2019
	
	
	
	
	
	
	
	
	
	
	
	
	
	
			
      
                181
            
            
                Posts
            
        
                205
            
            
                Kudos Received
            
        
                51
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 6113 | 10-19-2017 09:11 PM | |
| 2020 | 12-27-2016 06:46 PM | |
| 1579 | 09-01-2016 08:08 PM | |
| 1488 | 08-29-2016 04:40 PM | |
| 3990 | 08-24-2016 02:26 PM | 
			
    
	
		
		
		09-13-2022
	
		
		12:50 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @emaxwell   A few suggested additions to this excellent article:    1. The keystore password is not required to list the keystore via the keytool command.  When prompted for a password, you can just hit "enter" without entering a password.  The password would be needed if you want to export the actual privateCertEntry from the keystore which is not necessary in this procedure.    2. The ldap-provider configured in the login-identity-providers.xml file has two options for how to identify the authenticated ldap user:  <property name="Identity Strategy">USE_DN</property>  Above will use the full Distinguished Name (DN) as you mention in the article.      <property name="Identity Strategy">USE_USERNAME</property>   Above will use the string entered by user at the login window as the user identity.    3. Identity mapping properties in the nifi.properties file have three properties:  nifi.security.identity.mapping.pattern.<unique string>=<Java regex>
nifi.security.identity.mapping.value.<unique string>=<Resulting string when java regex matches, Java Regex capture groups can be used here>
nifi.security.identity.mapping.transform.<unique string>=<NONE, LOWER, UPPER>  The <unique string> can be anything but must be same exact string for all three properties.  Also patterns are checked in alphanumeric order against the identity strings. First matching pattern has its value and transform applied. So you want to make sure the more complex patterns come first to avoid having wrong pattern's value and transform being applied.  So a pattern like "^(.*)$" should definitely come last.    The "transform" property allows you to apply an all uppercase, all lowercase, or no transform to the resulting value string.    Thanks,  Matt 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-17-2022
	
		
		12:48 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 You are using an array, this is why you got null values. You have 2 options:   Change the expressions to $[0].ARRIVAL_DATE and $[0].BOOK_ID, and it will work  Read the file and split the array using the SplitJson processor - set JsonPath expression to: $, and connect the SplitJson processor to your EvaluateJsonPath processor and it should work.  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		05-05-2022
	
		
		04:36 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 As a general statement this is not right by any means. LDAP provides secure and encrypted authentication (encrypted user password and SSL/TLS communication) , together with user/group management. It's only the Hadoop stack does not support this and the two only autentication methods implemented for all the CDP components are the dummy simple auth (described above) and the Kerberos authentication (used in combination with PAM or LDAP for user/group mappings).  As an example, nothing less than Knox (the security gateway to HDP or CDP) implements full authenticacion using only LDAP (with TLS), and it only relies on Kerberos to authenticate a single service/proxy user to communicate with the rest of the cluster. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		02-17-2020
	
		
		06:29 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 RULE:[1:$1@$0](^.*@AD\.HORTONWORKS\.COM$)s/^(.*)@AD\.HORTONWORKS\.COM$/$1/g  I am not quiet sure if the brackets arountd (.*) are really necessary. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-07-2020
	
		
		11:56 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hello @saikrishna_tara @bbende @emaxwell .  Thanks for the solution and it worked well for me.     I am new to Nifi and got the same problem statement as @saikrishna_tara. I am able to make it till merge content, i can see my files are in parent flow files. but in parent name of files are uuid of the flow files and not the actual name of the file which is processed.  I need to put all the parent flow file's actual name via email processor to the outside world.  Please let me know in case more details are required.     Regards  Nitin 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-04-2019
	
		
		02:46 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hello, I'm looking your answer 3 years later because I'm in a similar situation :). In my company (telco) we're planning using 2 hot clusters with dual ingest because our RTO is demanding and we're looking for mechanisms to monitor and keep in sync both clusters. We ingest data in real-time with kafka + spark streaming, loading to HDFS and consuming with Hive/Impala. I'm thinking about a first approach making simple counts with Hive/Impala tables on both clusters each hour/half hour and comparing. If something is missing in one of the clusters, we will have to "manually" re-ingest the missing data (or copy it with cloudera BDR from one cluster to the other) and re-process enriched data. I'm wondering if have you dealt with similar scenarios or suggestions you may have. Thanks in advance! 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-19-2017
	
		
		09:11 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 @dsun  During the upgrade process, a component is supposed to be restarted after the hdp-select command has been run so it will pick up the new binaries. However, the component needs to shut down and start up after the hdp-select command has been run. That way it will report to Ambari that it's version has changed and what it's current state is. In the event that you get stuck (as you did) during the upgrade you can unwind the versioning with a process like this:   Make all pieces of the component are running  Run `hdp-select set` command on all nodes in the cluster to set the new version. Make sure you get all of the pieces for the component (e.g. hadoop-hdfs-namenode, hadoop-ndfs-journalnode, etc.)  Restart all processes for the component  Verify that the O/S processes are running with the proper version of jar files  Lather, rinse, and repeat for all components in the cluster   Once you have successfully gotten everything restarted with the proper bits, you should be able to manually finalize the upgrade with the following command to the Ambari Server:  ambari-server set-current --cluster=<custername> --version-display-name=HDP-2.6.2.0  If you get an error that components are not upgraded, you can check the components and hosts again. If everything seems ok, then you may need to tweak a table in the database. I ran into this when Atlas did not properly report the upgraded version to Ambari. NOTE: THIS SHOULD BE DONE WITH THE GUIDANCE OF HORTONWORKS SUPPORT ONLY  ambari=> SELECT h.host_name, hcs.service_name, hcs.component_name, hcs.version FROM hostcomponentstate hcs JOIN hosts h ON hcs.host_id = h.host_id ORDER BY hcs.version, hcs.service_name, hcs.component_name, h.host_name;
            host_name             |  service_name  |     component_name      |   version
----------------------------------+----------------+-------------------------+-------------
 scregione1.field.hortonworks.com | ATLAS          | ATLAS_CLIENT            | 2.6.1.0-129
 scregionm0.field.hortonworks.com | ATLAS          | ATLAS_CLIENT            | 2.6.1.0-129
 scregionm1.field.hortonworks.com | ATLAS          | ATLAS_CLIENT            | 2.6.1.0-129
 scregionm2.field.hortonworks.com | ATLAS          | ATLAS_CLIENT            | 2.6.1.0-129
 scregionw0.field.hortonworks.com | ATLAS          | ATLAS_CLIENT            | 2.6.1.0-129
 scregionw1.field.hortonworks.com | ATLAS          | ATLAS_CLIENT            | 2.6.1.0-129
 scregionm0.field.hortonworks.com | ATLAS          | ATLAS_SERVER            | 2.6.1.0-129
 scregionm1.field.hortonworks.com | DRUID          | DRUID_BROKER            | 2.6.2.0-205
 scregionm1.field.hortonworks.com | DRUID          | DRUID_COORDINATOR       | 2.6.2.0-205
 scregionw0.field.hortonworks.com | DRUID          | DRUID_HISTORICAL        | 2.6.2.0-205
 scregionw1.field.hortonworks.com | DRUID          | DRUID_HISTORICAL        | 2.6.2.0-205
 scregionw0.field.hortonworks.com | DRUID          | DRUID_MIDDLEMANAGER     | 2.6.2.0-205
 scregionw1.field.hortonworks.com | DRUID          | DRUID_MIDDLEMANAGER     | 2.6.2.0-205
 scregionm2.field.hortonworks.com | DRUID          | DRUID_OVERLORD          | 2.6.2.0-205
 scregionm2.field.hortonworks.com | DRUID          | DRUID_ROUTER            | 2.6.2.0-205
 scregionm2.field.hortonworks.com | DRUID          | DRUID_SUPERSET          | 2.6.2.0-205
 scregione1.field.hortonworks.com | HBASE          | HBASE_CLIENT            | 2.6.2.0-205
 scregionm0.field.hortonworks.com | HBASE          | HBASE_CLIENT            | 2.6.2.0-205
 scregionm1.field.hortonworks.com | HBASE          | HBASE_CLIENT            | 2.6.2.0-205
. . .  After verifying that you have, indeed, upgraded the components, a simple update command will set the proper version for the erroneous components and allow you to finalize the upgrade:  ambari=> update hostcomponentstate set version='2.6.2.0-205' where component_name = 'ATLAS_CLIENT';
UPDATE 6
ambari=> update hostcomponentstate set version='2.6.2.0-205' where component_name = 'ATLAS_SERVER';
UPDATE 1
  After cycling the Ambari Server, you should be able to finalize:  [root@hostname ~]# ambari-server set-current --cluster=<cluster> --version-display-name=HDP-2.6.2.0
Using python  /usr/bin/python
Setting current version...
Enter Ambari Admin login: <username>
Enter Ambari Admin password:
Current version successfully updated to HDP-2.6.2.0
Ambari Server 'set-current' completed successfully.
 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		06-12-2017
	
		
		01:00 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 what is the password for the above command? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		09-06-2017
	
		
		07:33 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi, you are right, the only solution is to change locale to english.  the code implementation to support multi-language is complex. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-26-2016
	
		
		06:52 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 What came up in a customer setting was the suggestion to not even use multitenancy. Instead, set up one shared realm so users can authenticate on all clusters. Then, assign access privileges using Ranger.  In older HDP versions this would create the problem of sharing the service principals and technical user accounts across clusters. I believe this has largely been solved in HDP 2.3. 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		 
        













