Atlas Elasticsearch Hook

[#1]

Hi

I'm looking to capture data linage in Atlas for our Java mapreduce jobs loading data from HDFS into Elasticsearch. Even an API example. We can track it from sqoop to hdfs but not the next step. App

[#2]

@David Sheard, Atlas lineage are generated using native types DataSet and Process (e.g. DataSet -> Process -> DataSet). If you can represent your pipeline of loading data from HDFS into elastic search using Process and DataSet types, Alas will help you generate the lineage for the above pipeline (there is no out of box solution for the above transformation pipeline). You can view an example about creating lineage for custom types here.

Hope this helps. If this answers your question please accept and up-vote the answer.

[#3]

Excellent, thanks for the link

Appreciated