NiFi - How to use JOLT to add json key:value dynamically to the Flowfile content?

[#1]

Input JSON content -

{ "exampledata": { "name": "Test", "age": null, "ver": null, "payloadType": "Text", "payloadData": "adsadsdsdsdsdsdsdsdsds", "sequenceNum": null, "timeStamp": "2017-09-22T12:07:29.968Z" } }

Output expected -

{ "exampledata": { "name": "Test", "age": null, "ver": null, "payloadType": "Text", "payloadData": "adsadsdsdsdsdsdsdsdsds", "sequenceNum": null, "timeStamp": "2017-09-22T12:07:29.968Z" "company" : "CompanyName" } }

company will be retrieved from Flowfile attribute and should be added to JSON Content.

Any help with JOLT Chain spec is much appreciated.

@Matt Burgess

[#2 · Accepted solution]

@Mahendra Hegde,

According to jira https://issues.apache.org/jira/browse/NIFI-3010

if you are using NiFi 1.2+ then we can use Attributes from the flowfile and add them to json.

I'm having Company attributes associated with the flowfile and value is also Company.

Input:-

{
"exampledata": {
"name": "Test",
"age": null,
"ver": null,
"payloadType": "Text",
"payloadData": "adsadsdsdsdsdsdsdsdsds",
"sequenceNum": null,
"timeStamp": "2017-09-22T12:07:29.968Z"
}
}

Jolt-spec:-

[{
 "operation": "shift",
 "spec": {
  "*": "&"
 }
}, {
 "operation": "default",
 "spec": {
  "Company": "${Company}"
 }
}]

Output:-

{
"exampledata": {
"name": "Test",
"age": null,
"ver": null,
"payloadType": "Text",
"payloadData": "adsadsdsdsdsdsdsdsdsds",
"sequenceNum": null,
"timeStamp": "2017-09-22T12:07:29.968Z"
},
"Company": "Company"
}

but this spec won't give the expected output as per your question, as we cannot add the attribute into main exampledata list.

Example1:-

by flattening out exampledata then we can add company attribute to the json message as follow.

Jolt-spec:-

[{
"operation": "shift",
"spec": {
"exampledata": {
"*": "&"
}
}
}, {
"operation": "default",
"spec": {
"Company": "${Company}"
}
}]

Output:-

{
"name": "Test",
"age": null,
"ver": null,
"payloadType": "Text",
"payloadData": "adsadsdsdsdsdsdsdsdsds",
"sequenceNum": null,
"timeStamp": "2017-09-22T12:07:29.968Z",
"Company": "Company"
}

Example2:-

If you are using prior version than NiFi 1.2, Then you need to use Replace text Processor with Below Properties.

Search Value

}\s+}

Replacement Value

,"company" : "${CompanyName}" } }

Replacement Strategy

Regex Replace

Evaluation Mode

Entire text

By using this method also we can add the attribute dynamically based on the flowfile attributes to the json message.

[#3]

@Shu - Thanks a lot, it is really helpful.