Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

NiFi convert json to csv with optional attributes

avatar
Rising Star

Hello,
I have a problem with converting json to csv.

I use ConvertRecord processor with attached JsonTreeReader and CSVRecordSetWriter.
The schema for both might look like this. Attributes in the json are optional.

 

 

{
"type": "record",
"name": "some_event",
"fields": [
{ "name": "date_time", "type": ["string","null"] },
{ "name": "id", "type": ["string","null"] },
{ "name": "is_success", "type": ["string","null"] },
{ "name": "username", "type": ["string","null"] },
{ "name": "username_type", "type": ["string","null"] }
]
}

 

for the input json like

 

{
"date_time": "2017-01-29T13:42:03.965Z",
"id": "20049015990001584400001"
"is_success": "TRUE",
}

 


where username and username_type is not set, the output does not correspond to my expectation

I get
2022-01-29T13:42:03.965Z|20049015990001584400001|true
instead of
2022-01-29T13:42:03.965Z|20049015990001584400001|true||
which should be the right format. Note the separator for null values at the end of the second record.

 

Do you have some hints, how to solve it? It should be sort of standard task.

 

The CSVRecordSetWriter is configured like this:

 

Jarinek_0-1647860858204.png

 


We use

Cloudera Flow Management (CFM) 2.0.4.3
1.11.4.2.0.4.3-1 built 01/06/2021 23:14:00 CET

Best regards
Jaro

1 ACCEPTED SOLUTION

avatar
Rising Star

I have eventually solved the issue. The Writter property "Schema Access Strategy" was misconfigured. It must be set to "Use 'Schema Text' Property" to really apply the given schema .

View solution in original post

1 REPLY 1

avatar
Rising Star

I have eventually solved the issue. The Writter property "Schema Access Strategy" was misconfigured. It must be set to "Use 'Schema Text' Property" to really apply the given schema .