Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

NiFi: Create the filename from the field value of csv flowfile

I have a csv flowfile with single record. I need to create its file name based on couple of column values in the csv file. Can you please let me know how we can do it by using the column name only not the position of the column as column position may change. Example 

CSV File

Name , City, State, Country, Gender

John, Dallas, Texas, USA, M

 

File name should be John_USA.csv

 

3 REPLIES 3

Expert Contributor

Hey,

 

This is the one option using Python script -

 

Just use ExecuteScript processor and use below script as script body -
(I am not Python expert, there should be better way to write below code, but this works)

 

import csv
import java.io
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback

global filename

class PyStreamCallback(StreamCallback):
    def __init__(self):
        pass

    def process(self, inputStream, outputStream):
        text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)

        lines = text.splitlines()
        reader = csv.reader(lines)
        headerList = list(next(reader))
        dataList = list(next(reader))
	global filename
        filename = dataList[headerList.index('Name')] + "_" + dataList[headerList.index('Country')] + ".csv"

        outputStream.write(bytearray(text.encode('utf-8')))

flowFile = session.get()
if (flowFile != None):
    flowFile = session.write(flowFile, PyStreamCallback())
    flowFile = session.putAttribute(flowFile, "filename", filename)
    session.transfer(flowFile, REL_SUCCESS)

 

Thanks

Mahendra

Expert Contributor

Hi,

This is one option using simple Python script -

(I am not python expert, there should be better way to write below code, but this works)

 

Just use ExecuteScript processor and add below code as script body - 

import csv
import java.io
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback

global filename

class PyStreamCallback(StreamCallback):
    def __init__(self):
        pass

    def process(self, inputStream, outputStream):
        text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)

        lines = text.splitlines()
        reader = csv.reader(lines)
        headerList = list(next(reader))
        dataList = list(next(reader))
	    global filename
        filename = dataList[headerList.index('Name')] + "_" + dataList[headerList.index('Country')] + ".csv"

        outputStream.write(bytearray(text.encode('utf-8')))

flowFile = session.get()
if (flowFile != None):
    flowFile = session.write(flowFile, PyStreamCallback())
    flowFile = session.putAttribute(flowFile, "filename", filename)
    session.transfer(flowFile, REL_SUCCESS)

 

 

Screenshot 2020-05-29 at 1.21.31 PM.png

 

Thanks

Mahendra

Expert Contributor

@vikrant_kumar24  - Have you found a better way to solve this ? If so would be curious to hear that!