question Re: NiFi Attribute expression in Support Questions

NiFi Attribute expression

lafi_oussama — Mon, 06 Nov 2023 16:46:39 GMT

Hello All

I need your Helps

I have a flowfile attribute with this expression attr1="${filename:find('OD_NCE.*.gz'):ifElse("/isod/brt_fin_fd_OD_NCE_pipe_sep","notfind")}"

I want to have the value of expression not the string

How I can execute this.

For Information : I read this expression from file and after this I convert the content to attributes

Re: NiFi Attribute expression

MattWho — Mon, 06 Nov 2023 19:08:56 GMT

@lafi_oussama

I am not clear in your ask here.
I assume you are using the above NiFi Expression language (NEL) statement in an Update Attribute processor?

Assuming the NiFi FlowFile has a filename=blah-OD_NCE-2023.gz

${filename:find('OD_NCE.*.gz') <-- resolves to true :ifElse("/isod/brt_fin_fd_OD_NCE_pipe_sep","notfind")} <-- since above is "true", the if else function would return "/isod/brt_fin_fd_OD_NCE_pipe_sep" and that woud be assigned to the "attr1" attribute on the outbound FlowFile.

If your filename does not match your java regular expression "OD_NCE.*.gz" (result in false) would result in attr1 = notfind

So I am not clear on what you are asking for:

I want to have the value of expression not the string

Can you provide an example of what you want to be assigned to FlowFile Attribute "attr1" and what you are actually getting?

This info is also not clear to me:

I read this expression from file and after this I convert the content to attributes

How are you reading expression from file? What exactly are you reading?
Why are you converting FlowFile content to FlowFile attributes? FlowFile Attributes reside within the NiFi's JVM heap. So you'll want to avoid this unless absolutely necessary and when you so, only extract the minimum needed.

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Re: NiFi Attribute expression

lafi_oussama — Tue, 07 Nov 2023 08:46:33 GMT

@MattWho

Hello

So I have a file that contains multiple properties like this :

${filename:find('OD_FUS.*.gz'):ifElse("/isod/brt_fin_fd_OD_FUS_pipe_sep",
${filename:find('OD_NCE.*.gz'):ifElse("/isod/brt_fin_fd_OD_NCE_pipe_sep",
${filename:find('DATA_Liste_Partenaire.*.gz'):ifElse("/baccarat/brt_fin_rn_partenaire","notfind")})})}

I developed python script that read this file and assign all this content on attribute attr1

And my flow read multiple files and will check if the file match any pattern from will give him destination in HDFS example:

filename=OD_FUS20231010.gz.20231010.csv

attr1 will have the value "/isod/brt_fin_fd_OD_FUS_pipe_sep"

and if the filename does not exists attr1 will have "notfind"

But now when I write the expression to attr1 he taked it as string he didn't execute it like this ${filename:find('OD_FUS.*.gz'):ifElse("/isod/brt_fin_fd_OD_FUS_pipe_sep",
${filename:find('OD_NCE.*.gz'):ifElse("/isod/brt_fin_fd_OD_NCE_pipe_sep",
${filename:find('DATA_Liste_Partenaire.*.gz'):ifElse("/baccarat/brt_fin_rn_partenaire","notfind")})})}

Re: NiFi Attribute expression

MattWho — Tue, 07 Nov 2023 17:41:33 GMT

@lafi_oussama

If I am understanding correctly, this sounds like expected behavior. You are writing a NiFi Expression Language (NEL) statement to an attribute via your python script so it is not being evaluated by the NEL process.

I am not clear on why you are using a python script here anyway.

As i understand your use case:
1. NiFi is ingesting files with various filenames.
2. Depending on the filename, you want to add an attr1 attribute to the FlowFile with a specific path value
3. If the filename does not match and value, you want to add an attr1 attribute to FlowFile with value "notfind".

This use case can easily be handled by a single UpdateAttribute processor through the use of the "Advanced" functionality without needing to develop and manage a python script. The "Advanced" functionality works like an If/Then/Else.
Would look something like this:

Outside the "Advanced" UI is the Else. Any dynamic attributes add here are added to FlowFile only if the same attribute was NOT added by a rule in the "Advanced" section.
So here you set attr1 = "notfind".

Inside the "Advanced" section you create one or more rules. Each rule has a Conditon (If) and an Action (then) sections. If the condition NEL resolves to "true" then actions are applied to FlowFile. So you set up a rule for each flowfile filename type. If none of teh rules match then the default attr = notfind is set.

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Re: NiFi Attribute expression

lafi_oussama — Wed, 08 Nov 2023 08:37:55 GMT

Hello @MattWho

My problem is like what you sad

But I have big number of expressions and can be added at any time so I want at every change go to the processor and add new expression, so my solution was to create file that contains all expressions and with python write its to attr1

Re: NiFi Attribute expression

lafi_oussama — Wed, 08 Nov 2023 10:51:03 GMT

Hello @MattWho

I found the solution is to add updateAttribute:

attr1=${attr1:evaluateELString()}