Skip to main content
VikasNayak07
Bronze ⭐️⭐️⭐️
March 11, 2022
Question

convert json to csv in nifi

Hi,

I am trying to convert json to csv in nifi. My json is something like below 

{"tokens":["12343455577666","2134421241241","124142141411445","21425654768686"]}

and my csv should be like below

tokens

12343455577666

2134421241241

124142141411445

21425654768686

Please help me. Thanks!

10 replies

Royles
Silver ⭐️⭐️
Silver ⭐️⭐️
March 11, 2022

There are a few posts on this topic.  The nearest to your need with a solution is
https://community.cloudera.com/t5/Support-Questions/NiFi-JSON-Array-split/m-p/158842#M121233

The key being using the SplitJson processor the updated doc reference is 
https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.9.0/org.apache.nifi.processors.standard.SplitJson/index.html

This will need to be used in combination with Reader > JsonPathReader > SplitJson > Writer

Translate
araujo
Platinum ⭐️⭐️⭐️
Platinum ⭐️⭐️⭐️
March 15, 2022

Hi, @VikasNayak07 ,

 

An easy way to achieve this is to use a JoltTransformRecord processor with a JsonTreeReader and a CSVRecordSetWriter. The following "Chain" transformation will produce the output that you want:

[
{
"operation": "shift",
"spec": {
"tokens": {
"*": "[&].token"
}
}
}
]

 

Cheers,

André

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Translate
-- Was your question answered? Please take some time to click on "Accept as Solution" below this post. If you find a reply useful, say thanks by clicking on the thumbs up button.
Kiranq
Silver ⭐️⭐️⭐️
October 2, 2023

@araujo 
 I am trying to implement your suggested way.
My json is :
{
"Id" : "some int",
"TenantId" : "some double value"
}

and I am using chain jolt transform method as:
[
{
"operation": "shift",
"spec": {
"*": {
"@(1,Id)": "[&1].Id",
"@(1,TenantId)": "[&1].TenantId"
}
}
}]

However, I am getting a error as:

Kiranq_0-1696263616383.png

Can you tell where I am going wrong?
Thanks. 

 

Translate
SAMSAL
Platinum ⭐️⭐️⭐️
October 2, 2023

@Kiranq,

from the error message I can see that your input seems to be in avro format. Are you sure the flowfile has valid json format? If not you can use ConvertAvroToJson processor before using the jolt transformation to convert avro to json. Also from the json you provided not sure your jolt spec will work. If you provide your json input and the expected output more accurately maybe I can help you with defining the correct jolt spec.

 

Translate
Kiranq
Silver ⭐️⭐️⭐️
October 2, 2023

@SAMSAL 
thank you for responding.

I am infact using/converting the output from ExecuteSql from Arvo to Json using ConvertAvroToJson processor. 
Below is the output of the same processor:

Kiranq_0-1696264800859.png



I am also wondering if I am providing the json format, its throwing error for avro. 
Can you suggest something?

Translate
SAMSAL
Platinum ⭐️⭐️⭐️
October 2, 2023

@Kiranq,

I have to look into the JoltTranformationRecord to see what configuration you have there vs what is expected. However since you are having a json input can you use the JoltTransformationJson processor instead or are you trying to convert the output json into different format?

Translate
Kiranq
Silver ⭐️⭐️⭐️
October 2, 2023

Thats right. I intend to convert the json using JoltTransfomationRecord into csv format and then save the file in my local machine. 

Translate
SAMSAL
Platinum ⭐️⭐️⭐️
October 2, 2023

Can you take screenshot of the JoltTranformRecord and the reader\write services settings?

Translate
Kiranq
Silver ⭐️⭐️⭐️
October 2, 2023

Here

JoltTransforRecord:

Kiranq_0-1696283131467.png

JsonTreeReader:

Kiranq_1-1696283171922.png

AvroReader:

Kiranq_2-1696283212507.png

Thanks.

Translate
SAMSAL
Platinum ⭐️⭐️⭐️
October 3, 2023

@Kiranq,

What I found is that the JoltTransformRecord expects only single record to work with hence the name. I noticed when I try to pass an array I was getting the error "...error transforming the first record", however if I pass just one json record it works. If you have an array of json\csv and you are looking to split and process each record individually then I would suggest that you split the records before the JoltTransformRecord. If you dont want to split the array then I recommend using JoltTranformJson first and then use Convert Record processor to convert to CSV.

Translate