Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to get all values with expression language in Apache Nifi Process Property (FetchElasticsearch)

avatar
Explorer

I am trying to create an Apache Nifi model that allows me to read in all my data from Elasticsearch and store it to a file. I have everything connected correctly but the issue I am having is the FetchElasticsearch process is requiring a document identifier (as it should) but I want to get every single item in the index it is searching not just a document with ID 1 for example. I know that Nifi and the process property supports expression language so I have tried simply using a regex expression that should match all characters which should just be:

${'*'}

but I get warning when I did this because the process actually looks for the literal document id that is * which of course does not exist. Below are screenshots so hopefully it can help with understanding my issue.

5263-nifi-flow.png

5264-elastic-prop.png

I am searching localhost:9300/tweet_library/tweet/(regex expression) so I want all of the documents in tweet_library. Any help is appreciated, thanks.

1 ACCEPTED SOLUTION

avatar
Master Guru

The FetchElasticsearch processor uses the Get API, which requires a single document identifier and doesn't support regular expressions. As an alternative, you can use InvokeHttp to call the Multi-Get API or the Search API, which give you more control over the retrieval of multiple documents.

View solution in original post

1 REPLY 1

avatar
Master Guru

The FetchElasticsearch processor uses the Get API, which requires a single document identifier and doesn't support regular expressions. As an alternative, you can use InvokeHttp to call the Multi-Get API or the Search API, which give you more control over the retrieval of multiple documents.