@leandrolinof  ListSFTP  and FetchSFTP are commonly used to fetch files from a remote server.  In order to use ListFile or FetchFile the location remote directory would need to be mounted as a local file directory. Matt ... View more

@PepeClaro Can you share your processor configuration and NiFi version being used?   Thanks, Matt ... View more

@ImranAhmed If I am understanding you correctly, you are trying to verify the content has correctly been modified before it is written out via your invokeHTTP processor.Inspecting that your dataflow is working as expected.  If you stop your InvokeHTTP processor after the ReplaceText processor, the FlowFile processed by the ReplaceText will queue in the connection.  You can then right click on the connection and list the queue.  From there you can view/download the content of that FlowFile post ReplaceText to verify it contains the expected modified content before the InvokeHTTP is started and writes it to your DB.   Thanks, Matt ... View more

@ImranAhmed  I am not clear ion what you are trying to do here. You are looking for some exact string and replacing with nothing? The search value is a java regex so what little i can see is looking for an exact string match which is being compared against the "entire text".   So entire text means entire contents of your FlowFile is being loaded into NiFi's JVM heap also.  Why "entire text" instead of "line-by-line"? ... View more

@wasabipeas  I can think of no way to force delete if it is blocking on a revision mismatch between nodes. Nothing here has anything to do with version control.  Is it always the same node reported in the pop-up message that fails to process the request? If so, have you verified the libs and version running on that one node match rest of cluster? If you go to the cluster UI and select "VERSIONS" tab, they all reflect same version? You could manually disconnect the one node that it keeps complaining about from the "NODES" tab. After it is disconnected, you could delete it from the cluster (Deleting the nodes does nothing flows or data on that node.  It will require a restart of that one node to get it to rejoin cluster). Once the node is removed form your cluster (temporarily), your cluster should reflect 10/10 connected nodes now in the status bar of the canvas UI. Check to see if your are still having revision issues with the process after reloading the page. If all looks good, you could access the filesystem of the currently disconnected and deleted node, stop the NiFi service on that node, and delete/rename the flow.xml.gz and flow.json.gz files.  Then start this node again. On startup, NiFi will inherit the flow from the cluster and in doing so get the cluster flows current revision for the problematic process group.  If problem persists, restart node that was deleted so that it rejoins the cluster. Then disconnected the currently elected cluster coordinator.  A new cluster coordinator will then be elected by zookeeper. Check to see if issue with process group is resolved. Reload your browser to force a page refresh. If issue is resolved, rejoin node to cluster via the cluster UI to see if issue returns.  If so, we at least know which is our problematic node.  You can of course, disconnect, delete, rename flow.xml.gz and flow.json.gz, and then restart node, just as we performed before so that flow is pulled from cluster on startup.  If issue still persists, there is something unique about this node.  Disk space ok?, any exceptions in logs?, while node may report same NiFi version, something different with contents of lib(s) folders (get a checksum and compare against other nodes). Hope this helps without needing to restart entire cluster, Matt ... View more

@prparadise  The NiFi MergeRecord assigns queued FlowFile on inbound connections to bins.  Bins can only contain "like FlowFiles". In order for two FlowFiles to be considered 'like FlowFiles', they must have the same Schema (as identified by the Record Reader) and, if the <Correlation Attribute Name> property is set, the same value for the specified attribute. Initial thoughts: 1. Perhaps your source FlowFiles are resulting in unique inferred schemas. The XMLRecordSetWriter can be configured with a schema write strategy such as "Set 'avro.schema' attribute" so that the output merged FlowFile has the schema added to an attribute (this would allow you to see the inferred schema on multiple FlowFiles to see if they match. 2. The min number of records per bin is set to 1 still.  When the Merge type processors execute that look at an inbound connection and allocate queued FlowFiles to bin(s).  At end of binning, it will see if any bin is eligible for merge. This processor can execute very fast and frequently.  Let's say that each time it executes, the inbound connection only contains 1 FlowFile. Since  min records per bin is 1, a bin with only one FlowFile would get merged.  Try setting the min records to a higher value.  Whenever you change the "min" settings, you should also set the "Max Bin Age" property.  This forces a bin to merge after the configured amount of time even if min values are not met. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@Chhavi  Your question is not very clear.   is there any way I can integrate clustered nifi for storing and fetching the value Storing and fetching what value? NiFi includes a couple Redis controller services: RedisConnectionPoolService  RedisDistributedMapCacheClientService  For example,  The RedisDistributedMapCacheClientService could be used by the PutDistributedMapCache NiFi processor to write values to Redis.  The FetchDistributedMapCache NiFi processor could be used to fetch values from Redis.   If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@ImranAhmed  Can you share a screenshot of your dataflow and the configuration of your replaceText processor? You mention xml. Is your source File that you are trying to perform replace text on an XML format file? If so, that is not a text based (ASCII) content file.  In that case, your configured search value is probably not matching on anything and this nothing gets changed in the original content binary content. NiFi is a content agnostic application.  This means that NiFi can ingest any type of data.  It does this by wrapping that content in to what NiFi calls a FlowFile.  A FlowFile consists of two parts: 1. FlowFile metadata/attributes - Stored in the NiFi flowfile_repository. It contains details about the content such as filename, size of content, location of stored content, and any other attributes added by NiFi components (processors, controller services, etc) as the FlowFile traverse these components in your dataflow. 2. FlowFile content - Stored in claim files with in the NiFi content_repository.  NiFi simple writes the binary contents to a claim and records the starting byte location and number of bytes of the content.  This way NiFi does not need to be able to read the content to move it through a dataflow. It becomes an individual component's code responsibility for knowing how to read the content of a FlowFile. So NiFi includes processor components for many different data types.  As far as XML content files, NiFi has limited native options (SplitXML, TransformXML, ValidateXML, XMLReader, and XMLRecordSetWriter).  The latter two are controller services that could be used by processors like convertRecord. There is also the possibility that one fo NiFi's scripting processor could be used where the user writes a script that can read an handle the specific content type.  There are execute processors that can execute and external command on the server where NiFi is running against the content of a FlowFile.  So if there is an external command service that can take content input and return modified content back. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@wasabipeas  What version of NiFi-Registry is being used as well? In your NiFi UI, search for component UUID (a8db3982-1350-1b8b-ffff-fffff988699d). What kind/type of component is it?  What is current state of the component (enabled, disabled, running, stopped, enabling, disabling, starting, stopping) Share screenshot of its current configuration. Thanks, Matt ... View more