Support Questions

Find answers, ask questions, and share your expertise

How to get all values with expression language in Apache Nifi Process Property (FetchElasticsearch)

avatar
Explorer

I am trying to create an Apache Nifi model that allows me to read in all my data from Elasticsearch and store it to a file. I have everything connected correctly but the issue I am having is the FetchElasticsearch process is requiring a document identifier (as it should) but I want to get every single item in the index it is searching not just a document with ID 1 for example. I know that Nifi and the process property supports expression language so I have tried simply using a regex expression that should match all characters which should just be:

${'*'}

but I get warning when I did this because the process actually looks for the literal document id that is * which of course does not exist. Below are screenshots so hopefully it can help with understanding my issue.

5263-nifi-flow.png

5264-elastic-prop.png

I am searching localhost:9300/tweet_library/tweet/(regex expression) so I want all of the documents in tweet_library. Any help is appreciated, thanks.

1 ACCEPTED SOLUTION

avatar
Master Guru

The FetchElasticsearch processor uses the Get API, which requires a single document identifier and doesn't support regular expressions. As an alternative, you can use InvokeHttp to call the Multi-Get API or the Search API, which give you more control over the retrieval of multiple documents.

View solution in original post

1 REPLY 1

avatar
Master Guru

The FetchElasticsearch processor uses the Get API, which requires a single document identifier and doesn't support regular expressions. As an alternative, you can use InvokeHttp to call the Multi-Get API or the Search API, which give you more control over the retrieval of multiple documents.