@Sean Dockery 1. The problem with G1GC in Java 8 is not unique to provenance. G1GC in Java 8 and earlier was still considered experimental. It was observed that G1GC had better performance with the larger heap sizes commonly used in NiFi setups, so early on it was the recommended GC for NiFI. While these bugs exist that can cause corruption in the in the Java heap space, we had not encountered the corruption prior to the introduction of the new WriteAheadProvenance implementation. While the G1GC issues have been resolved as of Java 9, those fixes were not back ported to earlier versions of Java. NiFi currently only supports Java 8, so we decided to move away from recommending using G1GC when using the new high performant WriteAheadProvenance implementation. https://wiki.apache.org/lucene-java/JavaBugs 2. The change of default provenance implementation from PersistentProvenance to WriteAheadProvenance only occurred recently and it appears no one changed the configuration in the bootstrap.conf at that time to comment out the G1GC line. It is also very likely we will again recommend G1GC once NiFi supports newer versions of Java where these G1GC issues have been addressed. 3. I have no answer for number 3, perhaps someone else can comment on that question in your query. Thank you, Matt If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more

@Adi Sutanto - Provenance replay inserts the FlowFile back on to the inbound connection to the processor. Since the GetFile processor does not have any inbound connections, there is no where to insert a FlowFile for replay. - As far as PutSFTP processor, you are correct that it generates "SEND" and "DROP" events. If you are auto-terminating the "failure" and/or "reject" relationships, you should see "DROP" events in provenance from which you can perform a "Replay". If these relationships are being routed via 1 or more connections, then your FlowFile should be sitting in one of those connections allowing you to redirect those FlowFile back to the PutSFTP to try again. - Thank you, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more

@Daniel Nguyen I see a mistake in your authorizers.xml: <property name="CN=localhost, OU=NIFI"></property> The above line should actually be: <property name="Node Identity 1">CN=localhost, OU=NIFI</property> Keep in mind that NiFi only builds the users.xml and authorizations.xml files from the authorizers.xml file configuration if those files do not already exist. So once you fix your authorizers,xml, you will need to delete or rename your existing users.xml and authorizations.xml files before starting NiFi. - Thanks, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more

@Bala S I would recommend using capture groups in your regex. Here is an example that will work for your example: I then replace with just the value from capture group 1 - Here is another example that is more generic (can have many ";<somevalue>" after first one: Here again I use capture group 1 as my replacement value. - Thank you, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link ... View more

@Vishwanath Joshi There are several processors you can use to write the attributes values to the content of the FlowFile which you can then send using putMongo processor. Here are a few examples - ReplaceText processor - You can use this to replace all content with the values from one or more FlowFile attributes using the NiFi expression language in the "Replacement Value" configuration property and the "Always replace" configuration option. - AttributesToJson processor - You use a Java regular expression to identify which FlowFile attributes will be used to generate json content and destination configuration of "flowfile content". - AttributesToCSV processor - You use a Java regular expression to identify which FlowFile attributes will be used to generate csv content and destination configuration of "flowfile content". - *** Link to details on NiFi Expression Language: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html - Thank you, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more

@Venkatesh AV - Just want to make sure we are using correct processor for what you want to do: The ReplaceText processor is used to replaceText in the content of the FlowFile. The UpdateAttribute processor could be used to replace text contained within an attribute of a FlowFile. - Assuming the FlowFile content is where you want to replace double quote with single quote, the following ReplaceText processor configuration will do that for you: - - Thank you, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more

@Michael Vikulin - Your nifi.properties file is configured to look for an Authoriuzer with the identifier Managed-authorizer. nifi.security.user.authorizer=managed-authorizer The shared authorizers.xml does not contain a "managed-authorizer". If you want to use the "file-provider" you need to update your nifi.properties file. - I also see that you are using ldap-provider for logging in to your NiFi. It is configured with: <propertyname="Identity Strategy">USE_USERNAME</property> This means that whatever string the user enters in the username login box will be parsed by any configured Identity.mapping.pattens configured in nifi.properties file and then resulting value string passed to authorizer. - So even once you fix your auithorizer.xml or nifi.properties file, You are likely going to send "admin" to your authorizer rather then the admin user's full DN. - Thanks, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more