How to get the list of all un-tagged Hive column in Atlas using Rest API command

[#1]

How to get the list of all un-tagged hive column/attributes in Atlas using Rest API command

Thank you,

Subash

[#2]

@subash sharma

AFAIK, there is no REST API to fetch untagged assets. You might want to consider writing small piece of code to filter untagged assets around REST API which returns all assets of specific type, in this case hive_column.

Hope this helps.

[#3]

@Ayub Khan, Agreed this can be achieved by code. In Atlas 0.6, I have seen that the length (key value pairs) of tagged entity (hive_column) are 10. Un-tagged column generally used to have 9 key value pairs. We can achieve this using a loop and the process will become time consuming. I was looking for some DSL search query or gremlin query which can give me the list of all untagged columns. I hope this defines my requirement.

[#4 · Accepted solution]

@subash sharma

Like @Ayub Khan mentioned currently there is no explicit REST end point to fetch all untagged entities, filtered by type (or by any other attributes). Alternatively you could use gremlin query to fetch untagged entities and add additional filters. But please note these queries may not be efficient for large number of entities and tags in the system. Also the result of gremlin query returns a list of vertices in its raw format and needs to be mapped to atlas structure.

http://localhost:21000/api/atlas/discovery/search/gremlin?query=g.V().has('__traitNames', T.notin, g.V().has('__type', 'typeSystem').filter{it.getProperty('__type.category').name() == 'TRAIT'}.'__type.name'.toSet()).has('__typeName', 'hive_column').toList()

[#5]

Hey @Sarath Subramanian, Thank you, can you please let me know any link from where i can get these gremlin queries.

[#6]

The below link have some gremlin queries (in TinkerPop2) we use in atlas

https://github.com/apache/incubator-atlas/blob/master/repository/src/main/java/org/apache/atlas/util...

Gremlin doc:

http://gremlindocs.spmallette.documentup.com

[#7]

Update for more recent versions of the platform: unfortunately the `/gremlin` endpoint has been removed from modern Atlas APIs, but you can use the special `_NOT_CLASSIFIED` identifier in the search DSL to meet this requirement.

So the query would simply be something like:

hive_column isA  _NOT_CLASSIFIED

---