Created 02-10-2019 08:50 PM
I am new to Apache atlas and I want to create Table and column level linage. I understood that linage can be created with the entity only using process. For a test purpose I have created custom types and entities under that type.
Now I want to show linage for Table and column relations. For Example. -
Table 1. Employee
Columns : Employee_ID, Employee_Name, Employee_Address
Table 2: Department
Columns: Employee_ID, Department_Name, Department_Code
I want to create Employee Table as an Entity and Columns of Employee Table as an Entities too. but columns will go as an attribute of Employee Table entity so i can create linage between Tables and Columns using Process. Same for the Department table.
Once it is created then I can create linage for Employee---->Process---->Department and also
Employee.Employee_ID----->Process---->Department.Department_Name
Please advice. Is it possible?
Created 02-11-2019 01:04 PM
Please take a look at https://github.com/apache/atlas/blob/master/addons/models/1000-Hadoop/1030-hive_model.json on how Hive is modeled in Atlas and based on that you can model your own type . hive_column is added as an attribute to hive_table type : https://github.com/apache/atlas/blob/master/addons/models/1000-Hadoop/1030-hive_model.json#L254-#L26... where hive_column is a type and entities can be created of hive_column type.
Created 02-11-2019 01:04 PM
Please take a look at https://github.com/apache/atlas/blob/master/addons/models/1000-Hadoop/1030-hive_model.json on how Hive is modeled in Atlas and based on that you can model your own type . hive_column is added as an attribute to hive_table type : https://github.com/apache/atlas/blob/master/addons/models/1000-Hadoop/1030-hive_model.json#L254-#L26... where hive_column is a type and entities can be created of hive_column type.
Created 02-11-2019 04:59 PM
@Sharmadha Sainath
Thanks for the guidance.. it works for me.