Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi: How can I update attributes(with Languaje Expression) inside flowfile.

Highlighted

NiFi: How can I update attributes(with Languaje Expression) inside flowfile.

New Contributor

How can I execute an SQL file loaded from the disk that contains references to attributes with Expression Language like "${environment}", It is loading and executing, but the references with Expression Language are not updating values.

File example:

query-in-file.png

In Flow: SelectHiveQL(filter table names) > ConvertArvoToJSON(parse list to json) > SplitJson(Split list by 'table_name') > EvaluatePathJson(to extract 'table_name' inside attribute 'tableName') > FetchFile( to load file from disk with path ${pathfile}/${tablename}.sql) > PutHiveQL (for execute the sql merge)

nifi-flow.png

The ParseException here is caused by PutHiveQL at moment to execute it:

exception-nifi-flow.png

Here exist a way to refresh the flowfile to update it the attribute references with Expression Language ?

Thanks in advance.

1 REPLY 1

Re: NiFi: How can I update attributes(with Languaje Expression) inside flowfile.

New Contributor

@Dennis Paitan if i understand your question. You need to replace some variables in sql file before executing them.

Say your sql file has statement as

select * from ${table} where name=${name} , and you want to replace "table" with attribute value "employee" and name with "Dennis" so actually query becomes

select * from employee where name="Dennis"

If this is your question, i would suggest to use ReplaceTextWithMapping processor. In this you can define a mapping file which will have mapping for your every match.

For e.g if my file is

select * from #table#

select * from #table1#

I have defined mapping file as

#table# Dennis

#table1# Paitan

and my Regex expression is #.*?#

Only thing is you have to define your regex properly to match your variable in sql file.

Don't have an account?
Coming from Hortonworks? Activate your account here