Support Questions

Find answers, ask questions, and share your expertise

NiFi: Create the filename from the field value of csv flowfile

avatar

I have a csv flowfile with single record. I need to create its file name based on couple of column values in the csv file. Can you please let me know how we can do it by using the column name only not the position of the column as column position may change. Example 

CSV File

Name , City, State, Country, Gender

John, Dallas, Texas, USA, M

 

File name should be John_USA.csv

 

3 REPLIES 3

avatar
Master Collaborator

Hey,

 

This is the one option using Python script -

 

Just use ExecuteScript processor and use below script as script body -
(I am not Python expert, there should be better way to write below code, but this works)

 

import csv
import java.io
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback

global filename

class PyStreamCallback(StreamCallback):
    def __init__(self):
        pass

    def process(self, inputStream, outputStream):
        text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)

        lines = text.splitlines()
        reader = csv.reader(lines)
        headerList = list(next(reader))
        dataList = list(next(reader))
	global filename
        filename = dataList[headerList.index('Name')] + "_" + dataList[headerList.index('Country')] + ".csv"

        outputStream.write(bytearray(text.encode('utf-8')))

flowFile = session.get()
if (flowFile != None):
    flowFile = session.write(flowFile, PyStreamCallback())
    flowFile = session.putAttribute(flowFile, "filename", filename)
    session.transfer(flowFile, REL_SUCCESS)

 

Thanks

Mahendra

avatar
Master Collaborator

Hi,

This is one option using simple Python script -

(I am not python expert, there should be better way to write below code, but this works)

 

Just use ExecuteScript processor and add below code as script body - 

import csv
import java.io
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback

global filename

class PyStreamCallback(StreamCallback):
    def __init__(self):
        pass

    def process(self, inputStream, outputStream):
        text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)

        lines = text.splitlines()
        reader = csv.reader(lines)
        headerList = list(next(reader))
        dataList = list(next(reader))
	    global filename
        filename = dataList[headerList.index('Name')] + "_" + dataList[headerList.index('Country')] + ".csv"

        outputStream.write(bytearray(text.encode('utf-8')))

flowFile = session.get()
if (flowFile != None):
    flowFile = session.write(flowFile, PyStreamCallback())
    flowFile = session.putAttribute(flowFile, "filename", filename)
    session.transfer(flowFile, REL_SUCCESS)

 

 

Screenshot 2020-05-29 at 1.21.31 PM.png

 

Thanks

Mahendra

avatar
Master Collaborator

@vikrant_kumar24  - Have you found a better way to solve this ? If so would be curious to hear that!