Support Questions

Find answers, ask questions, and share your expertise

Need more clarity on Nifi processors

avatar
Contributor

Can someone please give a working example of how processors work in nifi. What will be the output from one processor to another processor during the flow and how well we can access the floe attributes and flow content.

1 ACCEPTED SOLUTION

avatar
Master Guru

The output from one processor to another is a flow file, which consists of a map of attributes (key/value pairs) and a payload of bytes as the flow file content. The content could be raw binary data (an image, for example) or a text file in any format (JSON, XML, CSV to name a few). The content and attributes of a flow file are manipulated by the processors in different ways, the documentation for each processor will describe what attributes it reads and/or writes, as well as what operations it may perform on the processor. For example, the UpdateAttribute processor allows you to add attributes to (or delete them from) incoming flow files. Another example is SplitJson, which expects incoming flow files to have a JSON object as the flow file content, and then you configure the processor with a JSONPath expression pointing at an array within the object. Then the processor will split the original JSON object into individual JSON objects and send out flow files for each element in the array to the "split" relationship. It also sends the original incoming flow file to the "original" relationship. You can add connections between processors for the relationship(s) defined by the source processor.

I encourage you to read the Overview and Getting Started guides for more information on the concepts of NiFi (Flow Files, Processors, Connections, etc.). If you are looking for working examples, there is a set of templates available on the NiFi Wiki.

View solution in original post

3 REPLIES 3

avatar
Master Guru

The output from one processor to another is a flow file, which consists of a map of attributes (key/value pairs) and a payload of bytes as the flow file content. The content could be raw binary data (an image, for example) or a text file in any format (JSON, XML, CSV to name a few). The content and attributes of a flow file are manipulated by the processors in different ways, the documentation for each processor will describe what attributes it reads and/or writes, as well as what operations it may perform on the processor. For example, the UpdateAttribute processor allows you to add attributes to (or delete them from) incoming flow files. Another example is SplitJson, which expects incoming flow files to have a JSON object as the flow file content, and then you configure the processor with a JSONPath expression pointing at an array within the object. Then the processor will split the original JSON object into individual JSON objects and send out flow files for each element in the array to the "split" relationship. It also sends the original incoming flow file to the "original" relationship. You can add connections between processors for the relationship(s) defined by the source processor.

I encourage you to read the Overview and Getting Started guides for more information on the concepts of NiFi (Flow Files, Processors, Connections, etc.). If you are looking for working examples, there is a set of templates available on the NiFi Wiki.

avatar
Contributor

I have tried to delete the attribute from UpdateAttribute processor but it didn't delete it and also i tried to use AttributesToJson processor, expected only the attributes( which i specified in the attributes list) as a json but it gave me the whole attributes as a json. The behaviour of each processor is quite confusing and its not that user friendly in terms of configuring and using.

avatar
Contributor