Expression Language in Attribute Value

[#1]

Hello all,

I'm working on a metadata driven solution. The idea is to store metadata in a SQL table.

This table would contain something like this:

Filename	Filename Details
name 1	expression language
name 2	expression language

Is it possible to use that FilenameDetails attribute value and use that value in another processor? Something like evaluate Attribute? then I would use ${FilenameDetails} as a variable in processor to apply that expression language?

@mburgess  do you mind checking out my issue? 

Thanks,

Rosa

[#2]

First of all, based on your description it is hard to understand what you are trying to achieve. If you are looking to lookup/enrich your data with reference data, consider looking at the LookupRecord or LookupAttribute processors.

You could also load in the whole table in some format, parse it with an ExtractText or EvaluateJsonPath processor and use the attribute from there, but I don't think that is considered good practice in most cases.

[#3]

HI @DennisJaheruddi ,

thank you for your reply. 

let me try to explain. I want to read from my Metadata table and store the column values into attributes.

FilenamePattern	TimestampPattern	FilenameFilterRegex
indiana-YYYY-MM-DD THH-MM-SSZ-YYYYMMDDHHMMSS.gz	YYYY-MM-DD THH-MM-SSZ-YYYYMMDDHHMMSS	${http.param.path:matches('indiana-\d{4}-\d{2}-\d{2}T\d{2}-\d{2}-\d{2}Z-[0-9]{14}.gz')}

So, now I have my FF with the following attributes:

${FilenamePattern}=indiana-YYYY-MM-DD THH-MM-SSZ-YYYYMMDDHHMMSS.gz

${TimeStampPattern}=YYYY-MM-DD THH-MM-SSZ-YYYYMMDDHHMMSS

${FilenameFilterRegex}= ${http.param.path:matches('indiana-\d{4}-\d{2}-\d{2}T\d{2}-\d{2}-\d{2}Z-[0-9]{14}.gz')}

And, when I go and actually fetch the Flow File from HDFS, I want to Route on that FilenameFilterRegex attribute like this:

So, my questions is... can I literally evaluate the FilenameFilterRegex. Because, now, it will only evaluate it as a string, but it needs to be evaluated as regular expression.

Thanks,

Rosa

[#4]

It seems like you are looking for what most programming languages call an ‘eval’ function.

As suggested in this thread, there is no way to do this in NiFi, and it would likely also not be a good idea. A possible workaround would be to use a script, but at this point you should start wondering whether you are attacking the problem in the right way.

If you just want to evaluate the regex, consider putting it in your processor (rather than in the message itself), possibly you can route to multiple variants based on the characteristics of the data/attributes.

Otherwise, if you are able to build the regex for that specific message, consider executing it directly rather than storing the text of the regex and then sending it to NiFi.