In Apache Atlas, I am trying to model the data flow of different processes. The issue I am having is that some of these processes share common DataSets but I don't necessarily want the different processes I am modeling to appear to be connected to each other. For example, in this lineage model, I want to show that there is an input of an XML Data source file into a process that outputs and transferred to another computer. { "entity": { "typeName": "datasystem_datatransfer", "attributes": { "id":"b75af137-9279-4c73-be9f-0e37b686dde5", "qualifiedName": "b75af137-9279-4c73-be9f-0e37b686dde5@datasystem_datatransfer", "displayName": "Data Transfer Use Case 1", "inputs": [ { "uniqueAttributes":{"qualifiedName": "25b60fe5-891c-4c94-87ab-b075d838ec30@datasystem_datasource"}, "typeName": "datasystem_datasource" } ], "outputs": [ { "uniqueAttributes":{"qualifiedName": "21781e1b-4b94-435b-be0a-141776267c4e@datasystem_computer"}, "typeName": "datasystem_computer" } ], "description": "Data transfer from Data Source to Computer.", "name": "dataEgressUseCase1" } } } This will create a model like this:   datasystem_datasource --> datasystem_datatransfer --> datasystem_computer   I now have another process I want to model where I am using the same "datasystem_computer" but the process is a bit more complicated: { "entities":[ { "typeName": "datasystem_datatransfer", "attributes": { "id":"1305f6c4-f0da-4929-be21-dd0798dc2086", "qualifiedName": "1305f6c4-f0da-4929-be21-dd0798dc2086@datasystem_datatransfer", "displayName": "Data Transfer Use Case 2", "inputs": [ { "uniqueAttributes":{"qualifiedName": "c72375fb-34a5-4a22-895c-0d55435fdf26@datasystem_datasource "}, "typeName": "datasystem_datasource" } ], "outputs": [ { "uniqueAttributes":{"qualifiedName": "21781e1b-4b94-435b-be0a-141776267c4e@datasystem_computer"}, "typeName": "datasystem_computer" } ], "description": "Data Transfer from Data Source to PC.", "name": "dataEgressUseCase2" } }, { "typeName": "datasystem_datatransfer", "attributes": { "id":"307e6f84-41af-482e-8641-39fa258e709d", "qualifiedName": "307e6f84-41af-482e-8641-39fa258e709d@datasystem_datatransfer", "displayName": "Data Transfer Use Case 2.5", "inputs": [ { "uniqueAttributes":{"qualifiedName": "21781e1b-4b94-435b-be0a-141776267c4e@datasystem_computer"}, "typeName": "datasystem_computer" } ], "outputs": [ { "uniqueAttributes":{"qualifiedName": "5acddaca-6eb8-48f9-be75-fc757e442985@datasystem_datasource"}, "typeName": "datasystem_datasource" } ], "description": "Data Transfer from Data Source to PC to Another PC.", "name": "dataEgressUseCase2.5" } } ] } This should create a lineage diagram like:   datasystem_datasource --> datasystem_datatransfer --> datasystem_computer --> datasystem_datatransfer -->datasystem_datasource    The problem is that when I create this lineage, it changes the first lineage I created. They have different ID's so I am not sure why creating this second lineage would impact the first? I realize that they share the same datasystem_computer in one node, but they are different processes. What am I doing wrong? ... View more

Hello.  I would like to get more information about how to create Lineage in Apache Atlas.  Specifically, what is the right way to create lineage that includes multiple Processes and Data Sets.  Should I just be creating multiple Entities for the Process (using entity/bulk api endpoint)?  This seems to work, but it does create multiple Entities instead of just one complex entity:   { "entities": [{ "typeName": "mysystem_dataMovement", "attributes": { "id": "1305f6c4-f0da-4929-be21-dd0798dc2086", "qualifiedName": "1305f6c4-f0da-4929-be21-dd0798dc2086@mysystem_dataegress", "displayName": "Data Egress Use Case 1", "inputs": [{ "uniqueAttributes": { "qualifiedName": "c72375fb-34a5-4a22-895c-0d55435fdf26@mysystem_datasource" }, "typeName": "mysystem_datasource" }], "outputs": [{ "uniqueAttributes": { "qualifiedName": "b8e4ced9-f3f4-451a-8b24-3fa4d7970824@mysystem_computer" }, "typeName": "mysystem_computer" }], "description": "Data Egress from Data Source to Computer", "name": "dataEgressUseCase2" } }, { "typeName": "mysystem_dataMovement", "attributes": { "id": "307e6f84-41af-482e-8641-39fa258e709d", "qualifiedName": "307e6f84-41af-482e-8641-39fa258e709d@mysystem_dataMovement", "displayName": "Data Egress Use Case 2.5", "inputs": [{ "uniqueAttributes": { "qualifiedName": "b8e4ced9-f3f4-451a-8b24-3fa4d7970824@dbmesh_meshnode" }, "typeName": "mysystem_computer" }], "outputs": [{ "uniqueAttributes": { "qualifiedName": "5acddaca-6eb8-48f9-be75-fc757e442985@dbmesh_datasource" }, "typeName": "mysystem_datasource" }], "name": "dataEgressUseCase2.5" } } ] } ... View more

I am trying to figure out the Apache Atlas Common Library for my Spring Boot project. To be honest, I am having a really hard time finding any documentation on this library. But, I import it in my Pom file like this:   <dependency> <groupId>org.apache.atlas</groupId> <artifactId>atlas-common</artifactId> <version>2.2.0</version> </dependency>   I am trying to create a relationship between two entities. I know I can do this with the REST API, like this: { "typeName": "MyRealtionshipName", "end1": { "guid": "c69933db-f0b5-4f9a-99d9-b5f950f58b4f", "typeName": "myentitytype_1", "uniqueAttributes": { "qualifiedName": "obj1@id@myentitytype_1" } }, "end2": { "guid": "9ca8259f-673e-4dc5-80f2-67c8e6487cf4", "typeName": "myentitytype_2", "uniqueAttributes": { "qualifiedName": "obj1@myentitytype_2" } }, "label": "r:MyNewRelationship", "propagateTags": "NONE" }   But, I want to do this through the Apache Atlas library. I cannot find any documentation on how to do this. Can anyone assist and/or provide documentation? On the library side, I am able to create entities like this:   private final AtlasClientV2 atlasClient; AtlasService() { atlasClient = new AtlasClientV2(new String[]{"http://localhost:21000"}, new String[]{"admin", "admin"}); } public EntityMutationResponse createNewEntity(myOBj obj) throws AtlasServiceException { AtlasEntity newEntity= new AtlasEntity(); newEntity.setTypeName("entity_tpye"); newEntity.setAttribute("id", myOBj.getId()); newEntity.setAttribute("name", myOBj.getName()); AtlasEntity.AtlasEntitiesWithExtInfo entitesToCreate = new AtlasEntity.AtlasEntitiesWithExtInfo(); entitesToCreate.setEntities(Arrays.asList(newEntity)); EntityMutationResponse entitiesCreated = atlasClient.createEntities(entitesToCreate); //create relationship between this newEntity and a previously created entity with a given GUID } ... View more