@Vishwanath Joshi There are several processors you can use to write the attributes values to the content of the FlowFile which you can then send using putMongo processor. Here are a few examples - ReplaceText processor - You can use this to replace all content with the values from one or more FlowFile attributes using the NiFi expression language in the "Replacement Value" configuration property and the "Always replace" configuration option. - AttributesToJson processor - You use a Java regular expression to identify which FlowFile attributes will be used to generate json content and destination configuration of "flowfile content". - AttributesToCSV processor - You use a Java regular expression to identify which FlowFile attributes will be used to generate csv content and destination configuration of "flowfile content". - *** Link to details on NiFi Expression Language: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html - Thank you, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more

@Venkatesh AV - Just want to make sure we are using correct processor for what you want to do: The ReplaceText processor is used to replaceText in the content of the FlowFile. The UpdateAttribute processor could be used to replace text contained within an attribute of a FlowFile. - Assuming the FlowFile content is where you want to replace double quote with single quote, the following ReplaceText processor configuration will do that for you: - - Thank you, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more

@Michael Vikulin - Your nifi.properties file is configured to look for an Authoriuzer with the identifier Managed-authorizer. nifi.security.user.authorizer=managed-authorizer The shared authorizers.xml does not contain a "managed-authorizer". If you want to use the "file-provider" you need to update your nifi.properties file. - I also see that you are using ldap-provider for logging in to your NiFi. It is configured with: <propertyname="Identity Strategy">USE_USERNAME</property> This means that whatever string the user enters in the username login box will be parsed by any configured Identity.mapping.pattens configured in nifi.properties file and then resulting value string passed to authorizer. - So even once you fix your auithorizer.xml or nifi.properties file, You are likely going to send "admin" to your authorizer rather then the admin user's full DN. - Thanks, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more

@Mr Anticipation - The ERROR says you have a permissions issue. The user who owns the NiFi java process does not have permissions to navigate down the path /home/xxx/receive and/or does not have permissions to files you want to ingest. - Ambari by default creates the "nifi" service user account which is used to run NiFi. As such, that "NiFi" user must have access to traverse that directory path and consume the target file(s). - following command can be used to see what user owns the two nifi processes. # ps -ef|grep -i nifi - Thank you, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more

@Adam J The Remote Process Group (RPG) was not designed with any logic to make sure specific FlowFiles went to one node versus another. IT was designed to simply build a delivery model based on load on target NiFi cluster nodes. That delivery model will change potentially each time the lates cluster status is retrieved. - If you need to be very specific as to which node get a specific FlowFile, you best bet is to use a direct delivery dataflow design. The best option here is to have your splitText processor send to a routeOnContent processor that sends the split with URL 1/2 to one new connection and the flowfile with url 3/4 to another connection. Each of these connections would feed to a different postHTTP processor (this processor can be configured to send as flowfile). One of the would be configured to send to a listenHTTP processor on node 1 and the other configured to point at same listenHTTP processor on node 2. - You may want to think about this setup from a HA standpoint. If you lose either node 1 or 2, those flowfiles will just stack up and not transfer until the node is back online. at the same time the other urls continue to transfer. - Something else you may want to look into is the new load-balanced connections capability introduced in NiFi 1.8: https://blogs.apache.org/nifi/entry/load-balancing-across-the-cluster - There is a "Partition by Attribute" option with this new feature which would make sure flowfiles with matching attribute go to same node. While you still can't specify a specific node, it does allow similar flowfiles to get moved to same node. if node goes down you don't end up with an outage, but files with matching attributes will stay together going to different node that is still available. - Thanks, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more

@Nimrod Avni - The Standalone option is not ideal for setting up a NiFi cluster. Since the certificates generated are not signed by a Certificate Authority, the truststore will need to contain a trustedCertEntry for each certificate created. Adding additional nodes to a cluster would require going back and modifying the truststore on every node in the cluster. - The Client/Server mode allows you to standup a Certificate Authority (Server mode) that will be used to sign all the client certificate created (one for each NiFi node). When you stand up the Server a config.json is generated which can be used as input to the client mode operation. Because of this it is common that each of the client certificates are also generated from same server where the CA (server) was created/started. The client mode simply outputs a config.json file for each client certificate which simply provides the information needed to setup the relevant nifi.properties properties on each of your NiFi nodes. - It is safe to say that the structure will remain unchanged with a major NiFi release version. An external script could be used to update a nodes nifi.properties file from the output generated in the client mode config.json file. HDF for example already does this. If you choose to utilize the NiFi CA in HDF, it will take care of obtaining the client certificates and updating the nifi.properties on each node. This allows new client certificate to be generated on demand for each node. There is no option to configure NiFi to read these security parameters from the client mode generated config.json file. - Thank you, Matt - If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link. ... View more