Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

solr and atlas

avatar

Hello,

What is the exact relationship between atlas and solr? I thought metadata information write to Hbase.

1 ACCEPTED SOLUTION

avatar
Super Collaborator

@Anpan K

Yes , Atlas stores information in HBase. For fast information retrieval , the data need to be indexed and retrieved when queried.

Atlas uses Janus Graph. Janus graph has 2 types of indexes : composite and mixed indexes.

Composite indexes are supported with primary storage backend (HBase in this case), mixed graph indexes require indexing backend for full text search , numeric range search etc., Here , Solr is used by Atlas as indexing backend.

You may read about Janus Graph for more detailed information.

View solution in original post

2 REPLIES 2

avatar
Rising Star

Anpan K,
Atlas requires 3 services to work properly.

1. Kafka

2. Solr

3. HBase

Kafka is required so that changes done in the Hive metadata can be captured and a lineage can be created to be show in the ATLAS UI.

Solr is used to index that Atlas Data so that we can search the data in Atlas UI. It has three collections which make search happen. ( Full text index, edge index and vertex index)

HBase is used to store the actual data which is coming in the Atlas, In HDP 3 Janus graph has been launched while in earlier version it was Titan graph storage.

avatar
Super Collaborator

@Anpan K

Yes , Atlas stores information in HBase. For fast information retrieval , the data need to be indexed and retrieved when queried.

Atlas uses Janus Graph. Janus graph has 2 types of indexes : composite and mixed indexes.

Composite indexes are supported with primary storage backend (HBase in this case), mixed graph indexes require indexing backend for full text search , numeric range search etc., Here , Solr is used by Atlas as indexing backend.

You may read about Janus Graph for more detailed information.