solr and atlas

[#1]

Hello,

What is the exact relationship between atlas and solr? I thought metadata information write to Hbase.

[#2]

Anpan K,
Atlas requires 3 services to work properly.

1. Kafka

2. Solr

3. HBase

Kafka is required so that changes done in the Hive metadata can be captured and a lineage can be created to be show in the ATLAS UI.

Solr is used to index that Atlas Data so that we can search the data in Atlas UI. It has three collections which make search happen. ( Full text index, edge index and vertex index)

HBase is used to store the actual data which is coming in the Atlas, In HDP 3 Janus graph has been launched while in earlier version it was Titan graph storage.

[#3 · Accepted solution]

@Anpan K

Yes , Atlas stores information in HBase. For fast information retrieval , the data need to be indexed and retrieved when queried.

Atlas uses Janus Graph. Janus graph has 2 types of indexes : composite and mixed indexes.

Composite indexes are supported with primary storage backend (HBase in this case), mixed graph indexes require indexing backend for full text search , numeric range search etc., Here , Solr is used by Atlas as indexing backend.

You may read about Janus Graph for more detailed information.