Member since
02-13-2025
3
Posts
0
Kudos Received
0
Solutions
02-18-2025
11:13 AM
1 Kudo
@mridul_tripathi That is not exactly the dataflow I was trying to convey, but good attempt. This is what I was envisioning: It start with fetching the of files from "SFTP1" using the listSFTP and FetchSFTP processors. The ListSFTP processor will create a bunch of FlowFile attributes on the output FlowFile that can be used by the FetchSFTP to fetch the content and add it to the FlowFile. In the FetchSFTP processor you will specify the SFTP1 Hostname, Username, and Password. You will use NiFi Expression language to tell FetchSPT to fetch the specific content based in the FlowFile attributes created by ListSFTP: Next the FlowFile (now with its content from SFTP1) is passed to the CryptographicHashContent processor that will create a new FlowFile Attribute (content_SHA-256) on the flowFile with the content hash. Unfortunately, we have no control over the FlowFile attribute name created by this processor. Next The FlowFile is passed to an UpdateAttribute processor is used to move the (content_SHA-256) FlowFile to a new FlowFile attribute and remove the content_SHA-256 attribute completely so we can calculate it again later after fetch same file from SFTP2. I created a new FlowFile Attribute (SFTP1_hash) where I copied over the hash. Clicking the "+" will allow you to add a dynamic property. Next I pass the FlowFile to ModifyBytes processor to remove the content from the FlowFile. Now it is time to fetch the content for this same Filename from SFTP2 by using another FetchSFTP processor. This FetchSFTP processor will be configured with the hostname for SFTP2, username for SFTP2, and password for SFT2. We still want to use the filename from the FlowFile to make sure we are fetching the same file contents from SFTP2. So you can still use "${path}/${filename}" assuming both SFTP1 and SFTP2 use the same path. If not, you will need to set path manually (<some SFTP2 path>/${filename}). Now you pass the FlowFile to another CryptographicHashContent processor which will have the content fetched from SFPT2 for the same filename. At this point in time your FlowFile has a bunch of FlowFile attributes (including hash of both content from SFTP1 (SFTP1_hash) and SFTP2 (content_SHA256)and only the content from SFTP2. So you'll pass it Now it is time to compare those two hash attribute values to make sure they are identical using an RouteOnAttribute processor. Here will create a NiFi Expression Language (NEL) expression to make this comparison. Clicking the "+" will allow you to add a dynamic property. Each Dynamic property added in this property becomes a new relationship on the processor. ${content_SHA-256:equals(${SFTP1_hash})} This NEL will return the value/string from FlowFile's "content_SH256" attribute and check to see if it is equal to the value/string from the FlowFile's "SFTP1_hash" attribute. If true, the FlowFile will be routed to the new "Content-Match" relationship. If false, it will be routed to the exiting "unmatched" relationship. Here you can decide if just want to auto-terminate the "Content-Match" relationship or do some further processing. The Unmatched relationship will contain any FlowFiles where the content for two files of the same filename have content that did not match. The FlowFile will contain the content from SFTP2. Hope this helps. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more