question Re: solr and atlas in Archives of Support Questions (Read Only)

solr and atlas

ananthan_kathir — Fri, 29 Jun 2018 20:00:31 GMT

Hello,

What is the exact relationship between atlas and solr? I thought metadata information write to Hbase.

Re: solr and atlas

rbansal — Tue, 04 Dec 2018 20:57:14 GMT

Anpan K,
Atlas requires 3 services to work properly.

1. Kafka

2. Solr

3. HBase

Kafka is required so that changes done in the Hive metadata can be captured and a lineage can be created to be show in the ATLAS UI.

Solr is used to index that Atlas Data so that we can search the data in Atlas UI. It has three collections which make search happen. ( Full text index, edge index and vertex index)

HBase is used to store the actual data which is coming in the Atlas, In HDP 3 Janus graph has been launched while in earlier version it was Titan graph storage.

Re: solr and atlas

ssainath — Thu, 06 Dec 2018 21:19:09 GMT

@Anpan K

Yes , Atlas stores information in HBase. For fast information retrieval , the data need to be indexed and retrieved when queried.

Atlas uses Janus Graph. Janus graph has 2 types of indexes : composite and mixed indexes.

Composite indexes are supported with primary storage backend (HBase in this case), mixed graph indexes require indexing backend for full text search , numeric range search etc., Here , Solr is used by Atlas as indexing backend.

You may read about Janus Graph for more detailed information.