Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

NIfi ConvertJsonToSQL fails when Json has a array value and the column that needs to be mapped is varchar

avatar
Contributor

I have a json
 {
  "REQUESTID" : "379",
  "REQUESTTYPE" : "V",
  "REQUESTCONTEXT" : "B2B",
  "CARDNUMBER" : [ "5537290000000511", "5537290000000522" ],
  "EXPIRYDATE" : [ "08/20", "09/21" ],
  "EMAIL" : [ "[email protected]", "[email protected]" ],
  "PHONENUMBER" : "11234565555"
}
CARDNUMBER, EXPIRYDATE and EMAIL are varchar in sql table 
when Json is passed to covertToSQL processor, I am seeing empty string in the attribute for these columns in prepared statement (Insert)

1 ACCEPTED SOLUTION

avatar
Super Guru

Hi @Anderosn ,

The ConvertJsonToSQL according to the documentation will map fields only with simple type:

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache...

 

"...The incoming FlowFile is expected to be "flat" JSON message, meaning that it consists of a single JSON element and each field maps to a simple type..."

 

If you dont care about storing array fields as is, then you can use JoltTransformJson to convert array fields to concatenated string. The jolt spec to do this can be as the following:

 

[
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "CARDNUMBER": "=join(',',@(1,CARDNUMBER))",
      "EXPIRYDATE": "=join(',',@(1,EXPIRYDATE))",
      "EMAIL": "=join(',',@(1,EMAIL))"
    }
  }
]

This will produce the following output:

{
  "REQUESTID" : "379",
  "REQUESTTYPE" : "V",
  "REQUESTCONTEXT" : "B2B",
  "CARDNUMBER" : "5537290000000511,5537290000000522",
  "EXPIRYDATE" : "08/20,09/21",
  "EMAIL" : "[email protected],[email protected]",
  "PHONENUMBER" : "11234565555"
}

 

Which can be converted to SQL using the ConvertJsonToSQL  and all fields will be populated.

 

Hope that works.

If that helps please accept solution.

Thanks

 

 

 

View solution in original post

1 REPLY 1

avatar
Super Guru

Hi @Anderosn ,

The ConvertJsonToSQL according to the documentation will map fields only with simple type:

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache...

 

"...The incoming FlowFile is expected to be "flat" JSON message, meaning that it consists of a single JSON element and each field maps to a simple type..."

 

If you dont care about storing array fields as is, then you can use JoltTransformJson to convert array fields to concatenated string. The jolt spec to do this can be as the following:

 

[
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "CARDNUMBER": "=join(',',@(1,CARDNUMBER))",
      "EXPIRYDATE": "=join(',',@(1,EXPIRYDATE))",
      "EMAIL": "=join(',',@(1,EMAIL))"
    }
  }
]

This will produce the following output:

{
  "REQUESTID" : "379",
  "REQUESTTYPE" : "V",
  "REQUESTCONTEXT" : "B2B",
  "CARDNUMBER" : "5537290000000511,5537290000000522",
  "EXPIRYDATE" : "08/20,09/21",
  "EMAIL" : "[email protected],[email protected]",
  "PHONENUMBER" : "11234565555"
}

 

Which can be converted to SQL using the ConvertJsonToSQL  and all fields will be populated.

 

Hope that works.

If that helps please accept solution.

Thanks