Apache Atlas: Indexing Relationship data into Elasticsearch

[#1]

In have Atlas setup with Cassandra and ES. In ES entity data gets indexed in janusgraph_vertex_index. I want a way to add relationship data as well, so that I can support queries having filters on both Vertex and Edge attributes. How can I enable this use case using Atlas?

[#2]

Hello @suri07

To support search filtering across both vertex and edge attributes in Apache Atlas without breaking compatibility:

Denormalize the edge attribute into the vertex during ingestion.

• Add needed relationship info (like producer process type, owner, etc.) as attributes on the entity itself.

• Use /v2/entity/bulk to ingest this during lineage creation.

---

Bonus: Automating This

If you're using a metadata ingestion pipeline (e.g., from Airflow, Glue, custom ETL):

• Fetch related entities' properties.

• Inject them as attributes during entity creation.

• Keep a mapping logic (e.g., process’s jobType → dataset’s producing_job_type).