Skip to main content
Mazilinho
Mazilinho
Silver ⭐️
April 10, 2024
Question

Send the last CSV file from a NiFi flow to a separate processor

Hello!

I have a flow that extracts 100 CSV files from a SFTP sever. 

I need to identify the last CSV file in order to continue the flow after all the files are loaded in GCP bucket.

So, let me explain my scenario: I am extracting all 100 files from SFTP and load them in a GCP bucket. However, after the files are loaded, I want to redirect one file (preferably the last one) using the RouteOnAttribute in order to continue the flow for executing another processors.

Can someone help me please?

    3 replies

    DianaTorres
    Community Manager
    DianaTorres
    Community Manager
    April 10, 2024

    @Mazilinho Welcome to the Cloudera Community!

    To help you get the best possible solution, I have tagged our NiFi experts @mburgess @joseomjr  who may be able to assist you further.

    Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

    Translate
    Regards, Diana Torres, Senior Community Moderator
      Mazilinho
      MazilinhoAuthor
      Silver ⭐️
      April 11, 2024

      Thank you, Diana!  Another way to achieve this is to combine all flow files into one, but using MergeContent processor will not work, because Defragment option will fail, and the Bin-Packing option will merge all the flow-files into multiple files, and not only into one.

      Translate
        SAMSAL
        SAMSAL
        Platinum ⭐️⭐️⭐️
        April 13, 2024

        So how are you determining the last file? Based on what? is a file  created\updated date? or is it based on some sort order?

        Translate
        Terms & ConditionsAccessibility statement