Nifi - filter lookup table

[#1]

Hi, experts,

I need to enrich flowfile with data from database table, but before this table should be filtered based on attribute value. Is it possible to filter data within lookup processor? If not, please advise any solution for my task.

Best regards,
Alexey

[#2]

@Shalexey 

Can you provide an example of what you mean by filter?

The LookupRecord processor primary function is to extract one or more fields from a Record and looks up a value for those fields in a LookupService.

If you are trying to do a filter you can use another NiFi processor prior to the LookupRecord processor. Here are some examples below...

1. RouteOnAttribute

Routes flow files based on attribute values by evaluating expressions. It allows you to define conditions (e.g., ${status:equals('SUCCESS')}) to send data to different relationships, like success or unmatched.

2. RouteOnContent

Filters flow files based on patterns within the content. This processor is ideal for detecting specific keywords, tags, or values inside text, XML, or JSON data and routing them accordingly.

3. FilterCSV

Designed to filter rows in CSV files based on specific column values or positions. It removes unwanted data from the flow, only passing records that meet the defined criteria (e.g., status = 'active').

4. QueryRecord

Uses SQL queries to filter structured data (JSON, CSV, Avro) within flow files. You can select or discard records by writing SQL-like queries (e.g., SELECT * FROM FLOWFILE WHERE type = 'transaction').

[#3]

I'm looking up for ID in a table with following data:

id,key,attribute
1,same_value,attr1
2,same_value,attr2
3,same_value,attr3

As "key" has several records I need to filter the table by "attribute" in order to get one correct "id".

[#4 · Accepted solution]

Finally I replaced LookupRecord processor with group of Fork/JoinEnrichment.