Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Extract values from CSV and place it in a new column within the same CSV file using NiFi

Highlighted

Extract values from CSV and place it in a new column within the same CSV file using NiFi

I have a CSV file from which i need to extract two values like 'UutId & Test' and place it in the last column (filename) i generated by using 'CSVSetWritter' in UpdateRecord. How can i pull/extract values from same CSV file and place it in the last column of the same CSV file called 'filename' which is blank(without any values right now). If possible please tell what processor and configs to use?TIA

For reference i am attaching CSV file snapshot of value i want to add in the last column called 'filename' repeated in the whole CSV records.

86498-values-to-b-extracted.jpg

86497-lastcolumn.jpg

3 REPLIES 3
Highlighted

Re: Extract values from CSV and place it in a new column within the same CSV file using NiFi

Super Guru
@shraddha srivastav

Use UpdateRecord processor below configs

in CSVRecordSetWriter controller service add filename column with string type as last field in the avro schema.

UpdateRecord Configs:

enter image description here

Add new property in UpdateRecord processor as

/filename

concat(/UutId,/Test) //column names will be case sensitive

As we are using Record Path Value as Replacement Value Strategy

now update record processor will concat UutId,Test values to filename column value.

Refer to this link for more details regarding Update Record processor.

Example:

InputData:

UutId,Test
1,2

CsvReaderConfigs:

85621-csvr.png

CsvRecordSetWriter avro schema:

{
"namespace": "nifi",
"name": "balances",
"type": "record",
"fields": [
{"name": "UutId", "type": ["null", "string"]},
{"name": "Test", "type": ["null", "string"]},
{"name": "filename", "type": ["null", "string"]}
]
}

Configs:

85620-csvw.png

Ouput:

UutId,Test,filename
1,2,12

-

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.

Highlighted

Re: Extract values from CSV and place it in a new column within the same CSV file using NiFi

Super Guru
@shraddha srivastav

Stop Update Record and add some sample records after GetFile processor by listing the queue i.e. Input sample records like 10(not screenshots) and Expected output.

That would be helpful to recreated and resolve the issue..!!

Re: Extract values from CSV and place it in a new column within the same CSV file using NiFi

@Shu Please find attached the list queue after GetFile.

86504-listqueue.jpg

As soon as i start my UpdateRecord the List queue become blank. attaching the listqueue after UpdateRecord processor

86505-after.jpg

Expected output is to concatenate the row 9th value (FEBetaPack01) and 23rd row value (FE_ChargerSim_CurrentRipple)under column 'filename ' as (FEBetaPack01-FE_ChargerSim_CurrentRipple).

Please help. Thanks

Don't have an account?
Coming from Hortonworks? Activate your account here