Data transformation in Nifi

PriyankaMondal — Wed, 31 Jul 2024 12:05:55 GMT

Hi Team,
I want to achive the below mentioned transformation in Nifi. using any processor. Please help me to get this done.

sample input:
{
"date": "35 days 11:13:10.88",
"key1": "value1",
"keyToBeMapped1": "hostname.com",
"key2": "value2",
"key3": "value3",
"key4": "value4",
"keyToBeMapped2": "High Paging Rate",
"key5": "PAGING",
"keyToBeMapped3": "A high paging activity has been detected on host abc.lab.com. This could mean that too many processes are being run",
"Entity OID": "keyToBeMapped1",
"Parameter": "keyToBeMapped2",
"Description": "keyToBeMapped3"
}
Expected Output:
{
"date": "35 days 11:13:10.88",
"key1": "value1",
"keyToBeMapped1": "hostname.com",
"key2": "value2",
"key3": "value3",
"key4": "value4",
"keyToBeMapped2": "High Paging Rate",
"key5": "PAGING",
"keyToBeMapped3": "A high paging activity has been detected on host abc.lab.com. This could mean that too many processes are being run",
"Entity OID": "hostname.com",
"Parameter": "High Paging Rate",
"Description": "A high paging activity has been detected on host abc.lab.com. This could mean that too many processes are being run"
}

Regards,
Priyanka

Re: Data transformation in Nifi

florence0239 — Tue, 06 Aug 2024 11:35:09 GMT

@PriyankaMondal wrote:
Hi Team,
I want to achive the below mentioned transformation in Nifi. using any processor. Please help me to get this done. my lowes life

sample input:
{
"date": "35 days 11:13:10.88",
"key1": "value1",
"keyToBeMapped1": "hostname.com",
"key2": "value2",
"key3": "value3",
"key4": "value4",
"keyToBeMapped2": "High Paging Rate",
"key5": "PAGING",
"keyToBeMapped3": "A high paging activity has been detected on host abc.lab.com. This could mean that too many processes are being run",
"Entity OID": "keyToBeMapped1",
"Parameter": "keyToBeMapped2",
"Description": "keyToBeMapped3"
}
Expected Output:
{
"date": "35 days 11:13:10.88",
"key1": "value1",
"keyToBeMapped1": "hostname.com",
"key2": "value2",
"key3": "value3",
"key4": "value4",
"keyToBeMapped2": "High Paging Rate",
"key5": "PAGING",
"keyToBeMapped3": "A high paging activity has been detected on host abc.lab.com. This could mean that too many processes are being run",
"Entity OID": "hostname.com",
"Parameter": "High Paging Rate",
"Description": "A high paging activity has been detected on host abc.lab.com. This could mean that too many processes are being run"
}

Regards,
Priyanka

Hello,

You can achieve this transformation in NiFi using the JoltTransformJSON processor. Jolt is a JSON-to-JSON transformation library that allows you to specify transformations in a declarative way. Here’s how you can set it up:

Steps to Configure JoltTransformJSON Processor
Add the JoltTransformJSON Processor:
Drag and drop the JoltTransformJSON processor onto your NiFi canvas.
Configure the Processor:
Double-click the processor to open its configuration dialog.
Go to the Properties tab.
Set the Jolt Specification:
In the Jolt Specification property, you will define the transformation rules. Here’s the Jolt spec you need for your transformation:

[ { "operation": "shift", "spec": { "date": "date", "key1": "key1", "keyToBeMapped1": "keyToBeMapped1", "key2": "key2", "key3": "key3", "key4": "key4", "keyToBeMapped2": "keyToBeMapped2", "key5": "key5", "keyToBeMapped3": "keyToBeMapped3", "Entity OID": "@(1,keyToBeMapped1)", "Parameter": "@(1,keyToBeMapped2)", "Description": "@(1,keyToBeMapped3)" } } ]

Apply the Configuration:
Click Apply to save the configuration.
Connect the Processor:
Connect the JoltTransformJSON processor to the next processor in your flow.
Explanation of the Jolt Specification
The operation is set to shift, which means we are mapping fields from the input JSON to the output JSON.
The spec defines the mapping rules. For example, "Entity OID": "@(1,keyToBeMapped1)" means that the value of keyToBeMapped1 should be assigned to Entity OID in the output JSON.
Example Flow
GenerateFlowFile (to simulate input JSON)
JoltTransformJSON (with the above specification)
LogAttribute (to log the transformed JSON)
This setup should transform your input JSON to the expected output format.

Hope this will help you.
Best regards,
florence023

question Data transformation in Nifi in Support Questions

Data transformation in Nifi

Re: Data transformation in Nifi