To loop through the dataflow

Question

I have dataflow files which has to be routed on the basis of certain criteria.

It has multiple field like ("Customefied_23456: "Null") in the key: value pair. My query is how to filter out the flowfile which contain the some value for customfield ("customfield_33749": "this is required value",). i.e it does not contain null like ("Customefied_23456: "Null").

Please let me know how to proceed in this case.

thanks in advance.

MattWho · Accepted Answer

@midee

You could use a routeOnContent [1]processor to accomplish this. You would create a Java regex that matches only on customfield(s) where there is a string wrapped in quotes. If found, it routes entire FlowFile to the relationship created using the dynamic property's name.

RouteOnContent configuration:

I noticed in you example you have two customfield entries that do not have "null"

"customfield_10001": "This is required value",
"customfield_10002": "",

Based on my regex provided above, both of these would match resulting in FlowFile being routed to the "NotNull" relationship.
If i were to change the second .*? to .+? , then the customfield that contained only quotes would not match (just in case you only want to route when it is not null and not empty.

"customfield_.+?": ".*?",

versus

"customfield_.+?": ".+?",

If you found this addressed your query, please take a moment to login and click "accept" on this solution.
Thank you,

Matt

midee · Answer

@MattWho

Please advise me on this issue.

Issue:

In the flow file which contain at least customfield_ with some value other than null should be routed to another processor and the flowfile which has all null value should be moved to other processor.

thanks in advance!

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.