Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi: Create the filename from the field value of csv flowfile

NiFi: Create the filename from the field value of csv flowfile

I have a csv flowfile with single record. I need to create its file name based on couple of column values in the csv file. Can you please let me know how we can do it by using the column name only not the position of the column as column position may change. Example 

CSV File

Name , City, State, Country, Gender

John, Dallas, Texas, USA, M

 

File name should be John_USA.csv

 

3 REPLIES 3
Highlighted

Re: NiFi: Create the filename from the field value of csv flowfile

Rising Star

Hey,

 

This is the one option using Python script -

 

Just use ExecuteScript processor and use below script as script body -
(I am not Python expert, there should be better way to write below code, but this works)

 

import csv
import java.io
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback

global filename

class PyStreamCallback(StreamCallback):
    def __init__(self):
        pass

    def process(self, inputStream, outputStream):
        text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)

        lines = text.splitlines()
        reader = csv.reader(lines)
        headerList = list(next(reader))
        dataList = list(next(reader))
	global filename
        filename = dataList[headerList.index('Name')] + "_" + dataList[headerList.index('Country')] + ".csv"

        outputStream.write(bytearray(text.encode('utf-8')))

flowFile = session.get()
if (flowFile != None):
    flowFile = session.write(flowFile, PyStreamCallback())
    flowFile = session.putAttribute(flowFile, "filename", filename)
    session.transfer(flowFile, REL_SUCCESS)

 

Thanks

Mahendra

Highlighted

Re: NiFi: Create the filename from the field value of csv flowfile

Rising Star

Hi,

This is one option using simple Python script -

(I am not python expert, there should be better way to write below code, but this works)

 

Just use ExecuteScript processor and add below code as script body - 

import csv
import java.io
from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback

global filename

class PyStreamCallback(StreamCallback):
    def __init__(self):
        pass

    def process(self, inputStream, outputStream):
        text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)

        lines = text.splitlines()
        reader = csv.reader(lines)
        headerList = list(next(reader))
        dataList = list(next(reader))
	    global filename
        filename = dataList[headerList.index('Name')] + "_" + dataList[headerList.index('Country')] + ".csv"

        outputStream.write(bytearray(text.encode('utf-8')))

flowFile = session.get()
if (flowFile != None):
    flowFile = session.write(flowFile, PyStreamCallback())
    flowFile = session.putAttribute(flowFile, "filename", filename)
    session.transfer(flowFile, REL_SUCCESS)

 

 

Screenshot 2020-05-29 at 1.21.31 PM.png

 

Thanks

Mahendra

Highlighted

Re: NiFi: Create the filename from the field value of csv flowfile

Rising Star

@vikrant_kumar24  - Have you found a better way to solve this ? If so would be curious to hear that!

Don't have an account?
Coming from Hortonworks? Activate your account here