NIFI PuElasticsearchHttp Update - how does it work

[#1]

I need to push json data to Elasticsearch. I am using PuElasticsearchHttp processor. I have a scenario where the json data can hold insert, update or upsert. I tried to use IndexOperation = index for all these operations, then the ES index is getting updated with new value. But then I use IndexOperation = update it fails with "update requires valid identifier value with rom a flow file attribute.

[#2]

From the PutElasticsearchHttp documentation: "The name of the FlowFile attribute containing the identifier for the document. If the Index Operation is "index", this property may be left empty or evaluate to an empty value, in which case the document's identifier will be auto-generated by Elasticsearch. For all other Index Operations, the attribute must evaluate to a non-empty value."

So for "update", Elasticsearch needs to know which document to update. As a general rule, if the document has an identifier, you should use EvaluateJsonPath or something to extract the identifier into an attribute (let's call it "es.id" for this example), then in PutElasticsearchHttp you can set the Identifier Attribute property to "es.id".

[#3]

Thanks a lot Matt. I did figure it out finally. I introduced EvalJsonPath to extract the attribute and assign it to identifier. In case of single attribute, it worked. In my scenario i have multiple fields in my json - field1 and field2. I need to construct the _id field and send it to ES. I have used EvalJsonPath --> UpdateAttribute --> PutElastiSearchHttp. UpdateAttribute key is configured as _id and value is ${field1}_${field2}. During runtime I can see the attribute value getting constructed for example 111111_SmartAccount_700001, but I do not see the ES index getting populated. PutElastiSearchHttp processor is dropping this value. Any pointers?

[#4]

@mburgess @ljonnavi  Thank you!