Member since
01-31-2016
96
Posts
92
Kudos Received
20
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2676 | 02-11-2019 01:04 PM | |
3226 | 12-06-2018 01:19 PM | |
1863 | 08-23-2018 06:22 AM | |
1726 | 08-09-2018 11:29 AM | |
2457 | 03-29-2018 04:55 PM |
08-10-2018
05:53 AM
Eva Xiao , Messages are processed in the order they are received by Atlas. If message#1 has any error , only then message#2 is processed and action#3 specified by you can happen.
... View more
08-09-2018
11:29 AM
2 Kudos
Yes , it is important. Consider there are 2 events from Hive : 1. Rename an Hive table ( example : employee to employee_personal) 2. Add a column to the renamed Hive table. ( add address field to employee_personal) When Atlas Hive hook is configured , messages are sent for the above 2 events. Say , If message #2 is received first by Atlas first , employee_personal is not yet known to Atlas. Hence Atlas creates employee_personal hive_table entity with address field column + other columns. then , when message #1 is received , Atlas renames existing employee hive_table entity to employee_personal . Now , there are 2 employee_personal entities in Atlas, whereas in Hive , there is only 1 employee_personal table Hence , order is *very* important for Atlas being a Governance and Metadata management framework!
... View more
03-29-2018
04:55 PM
@Nicolas Schneider Please refer to http://atlas.apache.org/Import-Export-API.html
... View more
03-09-2018
04:09 PM
@Alisha Vaz POST the attached create-tag.txt to following REST API to create a tag named PII http://localhost:21000/api/atlas/v2/types/typedefs?type=classification After tag creation , get the GUID of the entity in Atlas which has to be associated to the tag. If you want to associate to an hive_table entity by name employee which is in default database , fetch the GUID of the entity using DSL search : typename = hive_table , query = where qualifiedName="default.employee@cl1" select __guid Fetch the GUID and post the associate-tag.txt to the following REST API: http://localhost:21000/api/atlas/v2/entity/bulk/classification In the associate-tag.txt file , in "entityGuids" json array ,replace the existing GUID with the GUID fetched from the DSL search query. Please refer to http://atlas.apache.org/api/v2/resource_TypesREST.html for more information on creating,updating,deleting tags and http://atlas.apache.org/api/v2/resource_EntityREST.html#resource_EntityREST_addClassification_POST for associating tags to entities.
... View more
03-07-2018
08:35 AM
@Satya Nittala 1. Please use correct port . By default , Atlas in non-SSL environment is configured to use 21000. 2. curl requires "@" for providing files. Example : -d @user/testplace.json 3. To update type , "PUT" (not POST. To create types use POST, to update types use "PUT") the JSON to http://atlashost:21000/api/atlas/v2/types/typedefs 4. As already mentioned , classification/tag best suits your requirement. Its highly recommended to use tags instead of updating types.
... View more
03-06-2018
12:50 PM
@Satya Nittala yes , POST the JSON body attached in the file to http://localhost:21000/api/atlas/v2/types/typedefs?type=classification In the tag definition , name is the name of the tag , and attributeDefs is an JSON array of attribute definitions. I have added expiry_date attribute of type date in the example. Once the tag is created , the tag can be associated to the hive_table entity by POSTing the attached tag-association.txt to http://localhost:21000/api/atlas/v2/entity/bulk/classification in tag-association.txt , "name" is the name of tag . attribute values can be provided in "attributes". entityGuids is the list of all the GUIDs of entity the tag should be associated to. In this array , you can provide the hive_table GUID.
... View more
03-06-2018
06:55 AM
@Satya Nittala For this requirement , please look at Classification. You can create a classification/tag with attributes. For example , create a tag named PI with required attributes like expiry date etc., and associate it to the hive_table entity. The attributes like columns , comments , aliases ,comment,createTime,db etc., are specific to Hive model. Information like Data custodian, Data owner & PI Information are not available in Hive. So it is not advisable to add such attributes to Hive model in Atlas. But,you may very well classify data based on tags - which is the recommended way. Once the table is associated to tag , you can query for the tag using search APIs , and it would list all the entities associated to the tag. For example , 1. Create a tag named PI with attribute expiry date of type date. 2. Associate the tag PI to the hive_table entity with date value for expiry date. 3.Now you can query for the tag PI with the particular expiry date. Please let me know if you need some more information on this.
... View more
03-02-2018
12:04 PM
@Satya Nittala hive_table is a type and fields you mentioned like aliases ,columns,comment,createTime,db etc., are attributes of hive_table. Type can be updated using PUT (http://atlas.apache.org/api/v2/resource_TypesREST.html#resource_TypesREST_updateAtlasTypeDefs_PUT). This requires fetching the type definition and updating with new attribute. For example , Following GET REST call is used to fetch the hive_table type definition : http://atlashost:21000/api/atlas/v2/types/entitydef/name/hive_table
After fetching the type definition , new attribute definition can be added in the attributeDefs array as {
name: "new_attribute",
typeName: "string",
isOptional: true,
cardinality: "SINGLE",
valuesMinCount: 0,
valuesMaxCount: 1,
isUnique: false,
isIndexable: false
} name : name of the new attribute typename : data type of the attribute isOptional : if the entity can be created without providing values for the attribute. (Note : updating a type with new mandatory attribute is not allowed. While updating , provide isOptional as True). and the updated JSON can be PUT to http://atlashost:21000/api/atlas/v2/types/typedefs For example , in the text file attached , I have added new attribute definition . GUID of the hive_table has to be modified based on your Atlas instance. Please let me know if you are stuck somewhere in this procedure. One question : hive_table is a defined type in Atlas.It has all attributes which will be required for maintaining hive meta data. May I know why you want to update it ? What is the new attribute you want to add ? could you please explain the use case behind it ?
... View more
03-02-2018
08:28 AM
2 Kudos
@Satya Nittala I hope you have enabled Atlas Hive hook settings. If yes , all updates to the hive table are captured by Atlas. When column is added in Hive, you can find the newly created hive_column entity in Atlas.
... View more
02-28-2018
02:20 PM
1 Kudo
@Anurag Mishra please check https://hortonworks.com/tutorial/tag-based-policies-with-apache-ranger-and-apache-atlas/ .
... View more