Support Questions

Find answers, ask questions, and share your expertise

Replacing Custom NiFi Processor for Converting JSON Content to Attributes

avatar
Rising Star

Hi Team,

I'm currently using a custom NiFi processor to convert JSON file content into flow file attributes, and I'm looking to replace it with a standard NiFi processor if possible.

Sample JSON Input:
The JSON files look like this (keys may vary per file):

{
"SourceFilePath": "/root/rdw/SrcFiles",
"SourceFilePattern": "Sales_data",
"SourceFile": "Sales_data_20241211.csv"
}


Current Setup:
I'm using a custom processor of type CreateFlatJSON (Bundle: NestedJSONToFlatJSON) to extract the key-value pairs from the JSON and assign them to attributes.

After processing, the flow file attributes look like:
SourceFilePath = /root/rdw/SrcFiles
SourceFile = Sales_data_20231111164536.csv
SourceFilePattern = Sales_data

These attributes are then used in downstream processors like so:
${SourceFilePath}/${SourceFile}

Current Flow:
FetchSFTP → ExtractText → CreateFlatJSON (Custom) → Downstream Processors
FetchSFTP: Retrieves the JSON file.
ExtractText: Reads the content into the flow file.
CreateFlatJSON: Parses JSON and converts key-value pairs to attributes.

Question
Is there a standard NiFi processor that can replace the custom CreateFlatJSON processor and achieve the same functionality — i.e., convert JSON content in the flow file into flow file attributes (with dynamic keys/values)?

1 ACCEPTED SOLUTION

avatar
Contributor

You can use a Groovy script with the ExecuteGroovyScript processor.

See this answer here for details

 

View solution in original post

3 REPLIES 3

avatar
Master Mentor

@s198 

I can't think of any NiFi stock processors that would create dynamic attributes nor can I figure out why you would so this.  I understand that you are using these "dynamic" ("${SourceFilePath}/${SourceFile}") attributes downstream in your dataflow, but to do so you are configuring these strings in that downstream processor meaning you need to know what they will be before even starting your processors.   If that is the case, these are not really dynamic since you need to know what they are to configure your downstream processors. 

If the following strings exist in all your source json, you can just declare them manually with EvaluateJsonPath processor to get the corresponding values from the FlowFile content:

SourceFilePath 
SourceFile 
SourceFilePattern


Can you share more info about your complete dataflow for better understanding?
What is happening downstream of your custom processor?

Thank you,

Matt

avatar
Contributor

You can use a Groovy script with the ExecuteGroovyScript processor.

See this answer here for details

 

avatar
Rising Star

Thank you @MattWho @linssab for the help.

The Groovy script with the ExecuteScript processor is working fine to convert JSON file content to flow file attributes.