@Priyank Rupera Please take a look at https://github.com/apache/atlas/blob/master/addons/models/1000-Hadoop/1030-hive_model.json on how Hive is modeled in Atlas and based on that you can model your own type . hive_column is added as an attribute to hive_table type : https://github.com/apache/atlas/blob/master/addons/models/1000-Hadoop/1030-hive_model.json#L254-#L266 where hive_column is a type and entities can be created of hive_column type. ... View more

@Anpan K Yes , Atlas stores information in HBase. For fast information retrieval , the data need to be indexed and retrieved when queried. Atlas uses Janus Graph. Janus graph has 2 types of indexes : composite and mixed indexes. Composite indexes are supported with primary storage backend (HBase in this case), mixed graph indexes require indexing backend for full text search , numeric range search etc., Here , Solr is used by Atlas as indexing backend. You may read about Janus Graph for more detailed information. ... View more

@Zakir Hossain This is because ZHXMRT user doesn't have authorization to read the zakirtestdb entity. If you log in as user who has read privileges (example : admin) , you should be able to click. ... View more

Yes , it is important. Consider there are 2 events from Hive : 1. Rename an Hive table ( example : employee to employee_personal) 2. Add a column to the renamed Hive table. ( add address field to employee_personal) When Atlas Hive hook is configured , messages are sent for the above 2 events. Say , If message #2 is received first by Atlas first , employee_personal is not yet known to Atlas. Hence Atlas creates employee_personal hive_table entity with address field column + other columns. then , when message #1 is received , Atlas renames existing employee hive_table entity to employee_personal . Now , there are 2 employee_personal entities in Atlas, whereas in Hive , there is only 1 employee_personal table Hence , order is *very* important for Atlas being a Governance and Metadata management framework! ... View more

@Nicolas Schneider Please refer to http://atlas.apache.org/Import-Export-API.html ... View more

@Alisha Vaz POST the attached create-tag.txt to following REST API to create a tag named PII http://localhost:21000/api/atlas/v2/types/typedefs?type=classification After tag creation , get the GUID of the entity in Atlas which has to be associated to the tag. If you want to associate to an hive_table entity by name employee which is in default database , fetch the GUID of the entity using DSL search : typename = hive_table , query = where qualifiedName="default.employee@cl1" select __guid Fetch the GUID and post the associate-tag.txt to the following REST API: http://localhost:21000/api/atlas/v2/entity/bulk/classification In the associate-tag.txt file , in "entityGuids" json array ,replace the existing GUID with the GUID fetched from the DSL search query. Please refer to http://atlas.apache.org/api/v2/resource_TypesREST.html for more information on creating,updating,deleting tags and http://atlas.apache.org/api/v2/resource_EntityREST.html#resource_EntityREST_addClassification_POST for associating tags to entities. ... View more

@Satya Nittala For this requirement , please look at Classification. You can create a classification/tag with attributes. For example , create a tag named PI with required attributes like expiry date etc., and associate it to the hive_table entity. The attributes like columns , comments , aliases ,comment,createTime,db etc., are specific to Hive model. Information like Data custodian, Data owner & PI Information are not available in Hive. So it is not advisable to add such attributes to Hive model in Atlas. But,you may very well classify data based on tags - which is the recommended way. Once the table is associated to tag , you can query for the tag using search APIs , and it would list all the entities associated to the tag. For example , 1. Create a tag named PI with attribute expiry date of type date. 2. Associate the tag PI to the hive_table entity with date value for expiry date. 3.Now you can query for the tag PI with the particular expiry date. Please let me know if you need some more information on this. ... View more