question Re: How to get the list of all un-tagged Hive column in Atlas using Rest API command in Archives of Support Questions (Read Only)

How to get the list of all un-tagged Hive column in Atlas using Rest API command

subash_sharma — Tue, 20 Jun 2017 15:36:42 GMT

How to get the list of all un-tagged hive column/attributes in Atlas using Rest API command

Thank you,

Subash

Re: How to get the list of all un-tagged Hive column in Atlas using Rest API command

apathan — Tue, 20 Jun 2017 15:53:59 GMT

AFAIK, there is no REST API to fetch untagged assets. You might want to consider writing small piece of code to filter untagged assets around REST API which returns all assets of specific type, in this case hive_column.

Hope this helps.

Re: How to get the list of all un-tagged Hive column in Atlas using Rest API command

subash_sharma — Tue, 20 Jun 2017 16:04:51 GMT

@Ayub Khan, Agreed this can be achieved by code. In Atlas 0.6, I have seen that the length (key value pairs) of tagged entity (hive_column) are 10. Un-tagged column generally used to have 9 key value pairs. We can achieve this using a loop and the process will become time consuming. I was looking for some DSL search query or gremlin query which can give me the list of all untagged columns. I hope this defines my requirement.

Re: How to get the list of all un-tagged Hive column in Atlas using Rest API command

ssubramanian — Tue, 20 Jun 2017 22:38:36 GMT

@subash sharma

Like @Ayub Khan mentioned currently there is no explicit REST end point to fetch all untagged entities, filtered by type (or by any other attributes). Alternatively you could use gremlin query to fetch untagged entities and add additional filters. But please note these queries may not be efficient for large number of entities and tags in the system. Also the result of gremlin query returns a list of vertices in its raw format and needs to be mapped to atlas structure.

http://localhost:21000/api/atlas/discovery/search/gremlin?query=g.V().has('__traitNames', T.notin, g.V().has('__type', 'typeSystem').filter{it.getProperty('__type.category').name() == 'TRAIT'}.'__type.name'.toSet()).has('__typeName', 'hive_column').toList()

Re: How to get the list of all un-tagged Hive column in Atlas using Rest API command

subash_sharma — Tue, 20 Jun 2017 22:45:18 GMT

Hey @Sarath Subramanian, Thank you, can you please let me know any link from where i can get these gremlin queries.

Re: How to get the list of all un-tagged Hive column in Atlas using Rest API command

ssubramanian — Tue, 20 Jun 2017 22:50:30 GMT

The below link have some gremlin queries (in TinkerPop2) we use in atlas

https://github.com/apache/incubator-atlas/blob/master/repository/src/main/java/org/apache/atlas/util/AtlasGremlin2QueryProvider.java

Gremlin doc:

http://gremlindocs.spmallette.documentup.com

Re: How to get the list of all un-tagged Hive column in Atlas using Rest API command

rjendoubi — Tue, 08 Mar 2022 11:25:22 GMT

Update for more recent versions of the platform: unfortunately the `/gremlin` endpoint has been removed from modern Atlas APIs, but you can use the special `_NOT_CLASSIFIED` identifier in the search DSL to meet this requirement.

So the query would simply be something like:

hive_column isA  _NOT_CLASSIFIED