Hello @Jobin George, @Bryan Bende, @Matt Clarke.. I have SSL configured in Nifi, we need to include one more authentication by adding login screen before getting into Nifi. Am trying to achive this by doing the following steps, but am not getting the login screen. could you please help me on this? FYI- I have installed Nifi on top LDAP server, I'm assuming that by default kerberos authentication is enabled on the server (Windows server 2012 R2) Is there any way to check, whether kerberos enabled on the server or not ( for Windows server 2012 R2) 1) Made below changes in Nifi.Properties file nifi.security.keystore=./conf/keystore.jks nifi.security.keystoreType=jks nifi.security.keystorePasswd=**password** nifi.security.keyPasswd=**password** nifi.security.truststore=./conf/truststore.jks nifi.security.truststoreType=jks nifi.security.truststorePasswd=**password** nifi.security.user.authorizer=file-provider nifi.security.user.login.identity.provider=kerberos-provider 2) Changed Login-identity-provider.xml <provider> <identifier>kerberos-provider</identifier> <class>org.apache.nifi.kerberos.KerberosProvider</class> <property name="Default Realm">******</property> <property name="Authentication Expiration">12 hours</property> </provider> What is Default Realm? Is it domain name or full computer name? 3) Below are the changes in authorizer.xml file <authorizer> <identifier>file-provider</identifier> <class>org.apache.nifi.authorization.FileAuthorizer</class> <property name="Authorizations File">./conf/authorizations.xml</property> <property name="Users File">./conf/users.xml</property> <property name="Initial Admin Identity">*******</property> <property name="Legacy Authorized Users File"></property> <property name="Node Identity 1"></property> </authorizer> ***Initial Admin Identity - Not sure, which user name I have to give. Currently I tried with some_user_name@domain_name ( Some_user_name -> created an user in Nifi user screen) I am not clear with few properties, what has to be provided over there. Your help is much appreciated! Awaiting for your response. ... View more

Hello @Matt Clarke @Shu & Everyone! I am trying to remove duplicates from CSV files. For that, I am using below processors. Extract text - To remove header from 2 flow files ( CSV file) Split Text - To split the text from flow files RouteOnContent - Route only the text, not the header HashContent - To Calculate hash value for all the flow files ( which got split) DetectDuplicate - To detect duplicate records and redirect non duplicate record to merge content. MergeContent - To merge the flow files again, based on correlation attribute name. In this case, sometimes am facing an issue, where a flow file is getting stuck between Detect duplicate & merge content. I have given max bin age as 5 mins, however even after 20 to 30 mins, it is not processing into merge content. Could someone share your inputs. (NOTE: This is standalone NiFi setup, not the cluster) Flow file expiration set to 5 hours. I have attached screen shots of Merge content configuration. One more thing, in production environment, we may get millions of record in two csv file. Is there any better way to find duplicate from CSV files, instead of splitting into text. If there is no other way, what configurations, I have to take care to avoid issues in production? Thanks, Iyyappan ... View more

Hello @Shu @Peter Greiff @Alex Gauthier @Timothy Spann & Everyone! Am trying to merge two CSV files, my ideal goal is to do an upsert ( If the record is new, then append it with existing record. if the already exists then, ignore that record). but what is happening, it just appending the all the records with existing records. Due to this am getting duplicate records. Any one having any idea? how to fix this? Your help is much appreciated. Thanks, Iyyappan ... View more