There are few issues with reusing the same entities after delete: 1. If the previously deleted entity has some tags - say PII associated with them, does the new table created inherit these tags? 2. If the new table is created with additional or lesser number of columns than the original, reusing tables will not help. If lineage cluttering is the issue, we can work towards adding filtering in lineage graph like exclude deleted entities from rendering. updating entities status from DELETED to ACTIVE doesn't help as only entity attribute updates are honored, but status, createTime, updateTime are all treated as system attributes of the entity. ... View more

@Laura Ngo, this is work in progress and we will be adding functionality to restrict certain tags from propagation using relationships. For more information on relationships: ATLAS-1690 ... View more

@Smart Data, can you tell some more details on how you create the process entity using Kafka? ... View more

@Carlos Guerra, some sample dsl like queries: hive_table where name like "sa?es*" hive_db where name like "R*" hive_db where hive_db.name like "R???rt?*" or hive_db.name like "S?l?s" or hive_db.name like "Log*" hive_db where hive_db.name like "R???rt?*" and hive_db.name like "S?l?s" and hive_db.name like "Log*" hive_table where name like 'sales*', db where name like 'Sa?es' ... View more

@Carlos Guerra, 'entity' attributes should contain only object id references. More information about the referred object id entity should be added in the 'referredEntities' section. For e.g. when creating hive_table entity, hive_column attributes are referred using its object ids in 'entity' section and detailed information about the hive column (like its own attributes) are defined in the referred entities section. If this answers your question please accept and upvote. ... View more

@Carlos Guerra, you can use the sample curl script to create hive_table entity using v2 APIs. Please note that all the referred entities like hive_column, hive_storagedesc are referred using negative long guids, also a valid db guid is specified for the db attribute. Hope this helps. curl -X POST \ http://localhost:21000/api/atlas/v2/entity \ -H 'authorization: Basic YWRtaW46YWRtaW4=' \ -H 'content-type: application/json' \ -d '{ "entity": { "guid": "-208942807557404", "status": "ACTIVE", "version": 0, "typeName": "hive_table", "attributes": { "owner": "hive", "temporary": false, "name": "test_table_0001", "qualifiedName": "my_db.test_table_0001@hdp_cluster", "columns": [ { "guid": "-208942807557402", "typeName": "hive_column" }, { "guid": "-208942807557401", "typeName": "hive_column" }, { "guid": "-208942807557400", "typeName": "hive_column" } ], "tableType": "MANAGED_TABLE", "sd": { "guid": "-208942807557403", "typeName": "hive_storagedesc" }, "partitionKeys": [ { "guid": "-208942807557405", "typeName": "hive_column" } ], "parameters": { "totalSize": "0", "rawDataSize": "0", "numRows": "0", "COLUMN_STATS_ACCURATE": "{\"BASIC_STATS\":\"true\"}", "numFiles": "0", "transient_lastDdlTime": "1497764442" }, "db": { "guid": "90a7d3af-873a-4c10-a815-069f2d47d490", "typeName": "hive_db" }, "retention": 0 } } , "referredEntities": { "-208942807557403": { "guid": "-208942807557403", "status": "ACTIVE", "version": 0, "typeName": "hive_storagedesc", "attributes": { "qualifiedName": "my_db.test_table_0001@hdp_cluster_storage", "storedAsSubDirectories": false, "location": "hdfs://localhost.localdomain:8020/apps/hive/warehouse/my_db.db/test_table_0001", "compressed": false, "inputFormat": "org.apache.hadoop.mapred.TextInputFormat", "outputFormat": "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat", "parameters": {}, "serdeInfo": { "typeName": "hive_serde", "attributes": { "serializationLib": "org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe", "parameters": { "serialization.format": "1" } } }, "table": { "guid": "-208942807557404", "typeName": "hive_table" }, "numBuckets": -1 } }, "-208942807557402": { "guid": "-208942807557402", "status": "ACTIVE", "version": 0, "typeName": "hive_column", "attributes": { "owner": "hive", "qualifiedName": "my_db.test_table_0001.col1@hdp_cluster", "name": "col1", "position": 0, "type": "int", "table": { "guid": "-208942807557404", "typeName": "hive_table" } } }, "-208942807557401": { "guid": "-208942807557401", "status": "ACTIVE", "version": 0, "typeName": "hive_column", "attributes": { "owner": "hive", "qualifiedName": "my_db.test_table_0001.col2@hdp_cluster", "name": "col2", "position": 1, "type": "string", "table": { "guid": "-208942807557404", "typeName": "hive_table" } } }, "-208942807557400": { "guid": "-208942807557400", "status": "ACTIVE", "version": 0, "typeName": "hive_column", "attributes": { "owner": "hive", "qualifiedName": "my_db.test_table_0001.col3@hdp_cluster", "name": "col3", "position": 2, "type": "string", "table": { "guid": "-208942807557404", "typeName": "hive_table" } } }, "-208942807557405": { "guid": "-208942807557405", "status": "ACTIVE", "version": 0, "typeName": "hive_column", "attributes": { "owner": "hive", "qualifiedName": "my_db.test_table_0001.col4@hdp_cluster", "name": "col4", "position": 5, "type": "string", "table": { "guid": "-208942807557404", "typeName": "hive_table" } } } } }' ... View more

@Saba Baig, you can search for any entities tagged with a particular tag name using the single quotes. for e.g. `Log Data` searches for all entities in atlas tagged as Log Data. Only classification name is supported in single quotes. ... View more