Support Questions
Find answers, ask questions, and share your expertise

NiFi: How to rename a getFile flowfile based on a ExecuteSQL_main query

Explorer

Hi, I am new to NiFi and want to rename a file based on a SQL query. The incoming jpg file has a GUID.jpg filename which I then extract the filename and query a table which returns the new filename. I then want to rename the file with this new filename. However when I use the ExecuteSQL_main processor the flowfile changes to the AVRO output and the original file is lost. The MERGE processor merges the content of the image incoming file with the avro query output which is not what I need.

Any help would be appreciated....

Thanks..

15 REPLIES 15

Explorer

Thanks James, this sounds groovy 🙂 .... I'm still playing with the merge solution so will try that later today.

Super Collaborator

@Zaheer N

Any luck? If you're having issues with either solution, let me know and I may be able to clarify. There were some details I left off in the first solution, particularly with the mergeContent (e.g. NumberOfMaxEntries=2, and I'm not sure what would happen if you have NumberOfMaxBins too low.).

@james.jones @Zaheer N I tried that and it works really well. Definitely Groovy !

Super Collaborator

Awesome. Can you accept the answer if that works out for you to close this out? Cheers.

Explorer

Apologies guys, I had to move on to another project and didn't get a chance to test the groovy solution. Seeing that it works as mentioned by Shishir I will accept the solution and try it first chance I get. Thanks once again for the effort. Cheers.

Super Collaborator

Thanks, @Zaheer N. FYI, one final option I'll throw out there is to use listFile (which was my original suggestion) and do a move/rename in Groovy. Here is a working groovy script in an ExecuteScript processor:

import org.apache.nifi.controller.ControllerService
import java.nio.file.Files
import java.nio.file.Paths
import java.nio.file.StandardCopyOption

/*

WARNING!!! This will OVERWRITE any existing file in the destination directory

REQUIRED FLOWFILE PROPERTIES:
     outDir             - directory name where the files will go (e.g. /tmp/junk/outdir)
     newFilename        - what you want the file named (e.g. mynew.jpg)
     changeFilenameAttr - optional boolean - default= 1, true or null sets origFilanem=${filename}
                          and filename=${newFilename}
                          value of false, 0 or n does not change the filename property.

     Also requires filename and orig.filename. These come from the getFile processor.

     Note - I wasn't able to getting properties defined in the ExecuteScript processor to work
            with Nifi Expression Language (e.g. referencing a property X=${filename} resolved to '${filename}'
            rather than the value of filename). The expressions were not evaluated for some reason.
*/


def flowFile = session.get()
if (!flowFile) return

log.info('*************** STARTING RENAME SCRIPT: ')


//*********
def origFilename = flowFile.getAttribute('filename')
def origPath = flowFile.getAttribute('absolute.path')

def sourceFilename = origPath + '/' + origFilename


def outDir = flowFile.getAttribute('outDir')
def newFilename = flowFile.getAttribute('newFilename')

def destFilename = outDir + '/' + newFilename
//*********

try {

   log.info('*************** MOVING FILE: ' + sourceFilename + ' TO ' + destFilename)

   def source = Paths.get(sourceFilename)
   def dest = Paths.get(destFilename)
   Files.move(source, dest, StandardCopyOption.REPLACE_EXISTING)

    if (flowFile.getAttribute('changeFilename')?.value?.toBoolean) {
       //This needs testing/verification
       flowFile = session.putAttribute(flowFile, 'origFilename', origFilename)
       flowFile = session.putAttribute(flowFile, 'filename', newFilename)
    }

    log.info('SUCCESSFULLY MOVED FILE: ' + sourceFilename + ' TO ' + destFilename)

    session.transfer(flowFile, REL_SUCCESS)
} catch(e) {
    log.error('FAILED moveFile.groovy: ' + sourceFilename + ' TO ' + destFilename, e)
    session.transfer(flowFile, REL_FAILURE)
}

And, thanks @Shishir Saxena for testing the previous script out.