Apache Atlas Types System V2

[#1]

1. Going through the Atlas Technical user guide I see the attribute isComposite, which I do not see in any of the new model definitions for V2. Is this attribute still valid? how can this be specified in v2?
2. I also see that constraints are defined in the Hive mode such as : ownedRef, mappedFromRef,foriegnKey, OnDelete. Where can I get the complete list of constraints and their meaning.
3. What is the relationship for? I see that Hive table refers to array of columns as its attributes and vice versa. and then there is a relationship also defined called hive_table_columns. what additional benefit does creating the relation provide?
4. How to create entities when handling compositions? If the table is composed of columns, will i define columns within the table creation request? An example creation of hive table and column (entities for V2) would be of great help. Thanks

@Ashutosh Mestry

[#2 · Accepted solution]

Let me attempt to answer all your questions:

1. Think of composite as composition in object oriented design world. An entity is part of the dependent entity. The lifetime of the child (contained entity) is determined by its parent. Child cannot exist without the parent.

2. I don't think there is a place where all of this is defined. We should definitely improve on our documentation. Here's link to code where this is defined. It is not ideal but better than nothing.

3. Relationships is a new concept. This implementation is only is master now. There is backward compatibility. Here's what the concept is: So far (before relationships) we would define associations between 2 entities as containment just by referencing one entity into another. See hive model in branch 0.8. Relationships merely allow you to capture this more comprehensively by modeling it separately. This will be apparent if you simply compare the hive model in master and branch-0.8.

4. For defining entities with composition, you could useAtlasEntity.AtlasEntityWithExtInfo. Add parent entity to the constructor and define the entities referenced as AtlasObjectIds. The creation API will take care of resolving the references. I am attaching JSON for reference. CURL entity creation is also added for reference.

Also see these object diagrams.

Hope this helps.

@Sarath Subramanian Thanks for your help in drafting this reply.

hive-table-sample.zipcreate-hive-table-entity.txt

[#3]

thank you @Ashutosh Mestry for such a detailed response. Helps alot! Unfortunately the hive entities are not created. I copied your hive table entities JSON in POSTMAN and i get the following error:

{
    "errorCode": "ATLAS-404-00-00A",
    "errorMessage": "Referenced entity 027a987e-867a-4c98-ac1e-c5ded41130d3 is not found"
}
<br>

[#4]

I copied the JSON contents from the create-hive-table-entity.txt to a file entity-create.json. Now I had to make a change to it so that the hive_db guid present in it had to be replaced with a guid of an already existing database. I replaced 90a7d3af-873a-4c10-a815-069f2d47d490 with 53ce4850-803e-457f-9f41-dfd01a761d9c.

I used this CURL command:

curl -k -X POST -u admin:admin -H "Content-Type: application/json" -H "Cache-Control: no-cache"  "https://localhost:21443/api/atlas/v2/entity" -d @../docs/entity-create.json

I was able to create the entities.

Note that my instance of Atlas has SSL enabled.

Can you please attempt this and let me know the results.

[#5]

Hi @Ashutosh Mestry I was using the json you provided in the Zip file. I tried the other file which works fine. The zip file is probably the response once the entities are created. Thanks

[#6]

That is correct. The ZIP file has the output and the TXT is the input.