1973
            
            
                Posts
            
        
                1225
            
            
                Kudos Received
            
        
                124
            
            
                Solutions
            
        My Accepted Solutions
| Title | Views | Posted | 
|---|---|---|
| 1823 | 04-03-2024 06:39 AM | |
| 2835 | 01-12-2024 08:19 AM | |
| 1570 | 12-07-2023 01:49 PM | |
| 2321 | 08-02-2023 07:30 AM | |
| 3200 | 03-29-2023 01:22 PM | 
			
    
	
		
		
		03-04-2024
	
		
		07:30 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @MvZ     The "file-login-provider" login identity-provider has never existed in any out-of-the-box release of Apache NiFi.    If you have created or downloaded some custom implementation of this provider. You would need to consult with that author in getting it to work.  Where did you obtain this provider from and what process did you follow to add it to your NiFi installation?    The exception you have shared simply tells you that during startup NiFi is loading the nifi.properties file and the property "nifi.security.user.login.identity.provider" is configured with "file-login-provider"; however, when NiFi parsed the login-identity-providers.xml configuration file, no provider with:  <identifier>file-login-provider</identifier>  was found in that configuration file.    I can't provide any guidance on this provider as I was unable to find anything online about what I am expecting is a custom add-on provider.    The out-of-the-box available authentication providers are found in the NiFi documentation here:  Apache NiFi 1.2x versions: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication  Apache NiFi 2.x versions: https://nifi.apache.org/documentation/nifi-2.0.0-M1/html/administration-guide.html#user_authentication    NiFi Authentication and Authorization are two different configurations and independent configurations.  Once you have chosen how you want to handle user authentication, you then move on to setting up user authorization:  https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#multi-tenant-authorization.    For file based authorization, NiFi offers two providers:  1. Older deprecated FileAuthorizer  2. The current StandardManagedAuthorizer    These providers are configured in the NiFi authorizers.xml file.  No direct useer policies get defined in the authorizers.xml file.  The FileAuthorizer or the FileAccessPolicyProvider referenced by the StandardManagedAuthorizer will generate the initial authorizations.xml file with the initial admin user configured in the provider chosen.  You would not typically manually generate or manipulate this file. Instead you would acces your NiFi's UI using that initial admin and define additional user authorizations directly via the NiFi UI.  Here is an example of what you would have in your authorizers.xml if using the StandardManagedAuthorizer:      <authorizers>
    <userGroupProvider>
        <identifier>file-user-group-provider</identifier>
        <class>org.apache.nifi.authorization.FileUserGroupProvider</class>
        <property name="Users File">./conf/users.xml</property>
        <property name="Legacy Authorized Users File"></property>
        <property name="Initial User Identity 1">ronald</property>
    </userGroupProvider>
    <accessPolicyProvider>
        <identifier>file-access-policy-provider</identifier>
        <class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
        <property name="User Group Provider">file-user-group-provider</property>
        <property name="Authorizations File">./conf/authorizations.xml</property>
        <property name="Initial Admin Identity">ronald</property>
        <property name="Legacy Authorized Users File"></property>
        <property name="Node Identity 1"></property>
    </accessPolicyProvider>
    <authorizer>
        <identifier>managed-authorizer</identifier>
        <class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
        <property name="Access Policy Provider">file-access-policy-provider</property>
    </authorizer>
</authorizers>     If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.  Thank you,  Matt 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-16-2024
	
		
		06:26 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi Timothy,  Thank you so much for helping. I also made some changes (adding index, change data types etc.,) for mysql database. Now, it ran much faster than before 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		01-12-2024
	
		
		12:33 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 I try using this custom processor but when I try it in tableName using ${db.table.name} like this         the result like this,         can you help me? its also when using ${filename} the result get the same ${filename} 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-11-2023
	
		
		12:40 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 Hi, @TimothySpann   Yes, our final upgrade version is 1.22.0 and there is not  ghosted processors.  We want to know the  basic principle of ListFile+Redis cache mapping. We found file state had some change that stored in "redis". The change is reflected in the precision of the file's modify time.  ps. We upgrade nifi on the same machine, time is still the same.       1.9.2  {'/mnt/EDA_DATA/NIFI/cloud/upgrade_test/ControlRate5/04594': {'timestamp': 1701853172000, 'size': 0}, '/mnt/EDA_DATA/NIFI/cloud/upgrade_test/ControlRate5/04593': {'timestamp': 1701853171000, 'size': 0}}  1.22.0  {'/mnt/EDA_DATA/NIFI/cloud/upgrade_test/ControlRate5/04594': {'timestamp': 1701853172307, 'size': 0}, '/mnt/EDA_DATA/NIFI/cloud/upgrade_test/ControlRate5/04593': {'timestamp': 1701853171506, 'size': 0}}    
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-07-2023
	
		
		01:49 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		1 Kudo
		
	
				
		
	
		
					
							 You can't listen for RSS, you have to call them since it's regular HTTP  https://github.com/tspannhw/FLaNK-TravelAdvisory     If there's no way to know when the page changes then you can't know.  you can read it once an hour and keep the entire results in a cache (like HBase) and if it doesn't change throw it away 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		12-07-2023
	
		
		12:27 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @Coordinador,  have any of the replies helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		10-13-2023
	
		
		06:53 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 it's very hard to format a long call like that without wrapping it.   also you will need to fully list out directories so not /test.pdf but /the/actual/location/even/if/it/s/giant/test.pdf.  it may work.   you may also need to do /usr/bin/sshpass.   it is always easier to just write a simple runme.sh with the text in it, make it executable and test it.    also make sure nifi user has permissions to sshpass and if you are running on a cluster that all machines have that binary and have permissions 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		09-20-2023
	
		
		06:58 AM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							 @kellerj, Has the reply helped resolve your issue? If so, can you please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future?  If you are still experiencing the issue, can you provide the information @TimothySpann  has requested? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		09-17-2023
	
		
		10:27 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
				
		
	
		
					
							  > ListFile will lose tracking.  This is beyond my knowledge. According to the introduction of ListFile, the tracking entity will be stored in redis. Why is it lost when I only upgrade nifi? 
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		
			
    
	
		
		
		08-06-2023
	
		
		06:07 PM
	
	
	
	
	
	
	
	
	
	
	
	
	
	
		
	
				
		
			
					
	
		2 Kudos
		
	
				
		
	
		
					
							 You need to look at CDC tools like Cloudera Kafka Connect with Debezium or Cloudera SQL Streaming Builder with Debezium.     Kafka Connect  https://docs.cloudera.com/runtime/7.2.17/kafka-connect/topics/kafka-connect-connector-debezium-oracle.html     SSB  https://docs.cloudera.com/csa/1.10.0/how-to-ssb/topics/csa-ssb-cdc-connectors.html        Or you can do Oracle GoldenGate with Kafka.     I have some examples in this article  https://medium.com/cloudera-inc/cdc-not-cat-data-capture-e43713879c03    
						
					
					... View more
				
			
			
			
			
			
			
			
			
			
		 
        













