Nifi AttributesToJSON makes everything strings

[#1]

I have JSON messages in the flowfiles, and I'm putting a subset of the JSON elements into attributes using EvaluateJSONPath.   The issue is that when I use AttributesToJSON, all the elements get wrapped in quotes, so the JSON fields that are/were arrays now are just strings that happen to have square brackets in them, and quotes around the strings in the array are escaped.   How do I preserve the "non-stringiness" of my JSON through these operations?

Input JSON:

{"customer_id": 1234567, "vckey_list": ["test value"]}

I pull both of those fields into attributes using EvaluateJSONPath, and then user AttributesToJSON with content-flowfile and this is the output.   Obviously the strings within the array are escaped because the array itself is quoted, as is the thing that was a number.

Output JSON after 2 operations:

{"customer_id": "1234567", "vckey_list": "[\"test value\"]"}

My desired output would be identical to the original.  

{"customer_id": 1234567, "vckey_list": ["test value"]}

I'm sure I've missed something simple to make these operations commute.

[#2]

I have the same problem, any chance you found a solution?

[#3]

@craignelson7007 

Since you are starting with JSON content which you are extracting bits of into FlowFileAttributes just to write them back to json, have you considered skipping this completely and just using the NiFi JoltTransformJson processor to modify the existing json content instead?

Here is a article with more details on using the JoltTransformJson processor:
https://community.cloudera.com/t5/Community-Articles/Jolt-quick-reference-for-Nifi-Jolt-Processors/t...

Otherwise:

NiFi stores all FlowFile attributes as strings.
The json you have written out in a FlowFile Attribute is going to be stored as a string in the attribute key 

vckey_list = ["test value"]

The AttributesToJson processor builds a flat json from those attribute strings, so it is going to wrap the entire json in quotes and escape the embedded quotes within that string with a backslash.

You could use a couple replaceText processors after your AttributesToJson processor to remove the unwanted quotes and backslashes from the newly created content.

Here is another reference around this processor limitation:
http://apache-nifi.1125220.n5.nabble.com/Nesting-and-AttributesToJSON-td3575.html

Hope this helps,

Matt

[#4]

I was able to do it using replaceTextProcessor, I see that JoltTransformJSON is a fancier way to do it. I will give it a try, thanks!

[#5]

Not all credit goes to me but I was pointed to a better simpler way to achieve this. There are 2 ways.

Solution 1 - and the simplest and elegant
Use Nifi JoltTransformJSON Processor. The processor can make use of Nifi expression language and attributes in both left or right hand side of the specification syntax. This allows you to quickly use the JOLT default spec to add new fields (from flow-file attributes) to a new or existing JSON.

Ex:

{"customer_id": 1234567, "vckey_list": ["test value"]}

 both of those fields values are stored in flow-file attributes as a result of a EvaluateJSONPath operation. Assume "customer_id_attr" and ""vckey_list_attr". We can simply generate a new JSON from those flow-file attributes with the "default" jolt spec and the right hand syntax. You can even add addition expression language functions to the processing

[
  {
    "operation": "default",
    "spec": {
      "customer_id": ${customer_id_attr},
      "vckey_list": ${vckey_list_attr:toLower()}
    }
  }
]

This worked for me even when storing the entire JSON, path of "$", in a flow-file attribute.

Solution 2 - complicated and uglier
Use a sequence Nifi ReplaceText Processor. First use a ReplaceText processor to append the desired flow-file attribute to the file-content.

If you are generating a totally new JSON, this would do it. If you are trying to modify an existing one, you would need to first append the desired keys, than use ReplaceText again to properly format as a new key in the existing JSON, from 

{"original_json_key": original_json_obj}{"customer_id": 1234567, "vckey_list": ["test value"]}

to

{"original_json_key": original_json_obj, "customer_id": 1234567, "vckey_list": ["test value"]}

using

Then use JOLT to do further processing (that's why Sol 1 always makes sense)

Hope this helps, spent about half a day figuring out the 2nd Solution and was pointed to Solution 1 by someone with more experience in Nifi