question How to fetch total numbers of tables available in a Schema using Rest API search techniques ? in Archives of Support Questions (Read Only)

How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

subash_sharma — Fri, 09 Jun 2017 16:27:33 GMT

How to get the list pf all the tables available in a particular schema. This Rest API command generally gives the GUID and total count of all tables available in Atlas.

http://hostname:port/api/atlas/entities?type=hive_table

Can we apply some sort of filter on database or schema ?

Thank you,

Subash

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

amestry — Sat, 10 Jun 2017 01:41:42 GMT

Right now there isn't a way to sort the output. The consumer of this API will have to do the sorting.

Alternative to this would be to use DSL query. See this page for some samples.

DSL search converts the query into Gremlin query and then fires it on to the database.

Your query would be like this: hive_table orderby name

REST call would be: http://localhost:21000/api/atlas/v2/search/dsl?limit=25&excludeDeletedEntities

Hope this helps.

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

anaik — Sat, 10 Jun 2017 03:13:54 GMT

Like @Ashutosh Mestry said this can be achieved through DSL or Gremlin calls.

from DB select count() as 'count'
from Table select count() as 'count'

Also remember that the count queries are pretty expensive and might take a while to return results as it requires a major graph scan.

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

subash_sharma — Mon, 12 Jun 2017 13:31:09 GMT

Hey @Ashutosh Mestry,

I should have mentioned Atlas version as well. I am currently working on Atlas 0.7 and I guess the rest API request which you have mentioned will work on Atlas 0.8. Are there any similar rest API request available for Atlas 0.7 ?

Even if i will search on hive_table orderby name , It will list down all the tables from all schema. I am searching a way to list down list of all tables of a particular schema.

Thank you,

Subash

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

ssainath — Mon, 12 Jun 2017 15:16:26 GMT

@subash sharma

DSL query : hive_table where db.name="default" lists down all the hive tables in "default" database.

V1 Query which can be used in Atlas 0.7 :

/api/atlas/discovery/search/dsl?query=hive_table+where+db.name%3D%22default%22

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

subash_sharma — Mon, 12 Jun 2017 15:25:58 GMT

Bingo, This is what i was looking for !

Thank you @Sharmadha Sainath

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

ssainath — Mon, 12 Jun 2017 15:43:47 GMT

Welcome @subash sharma . Please accept the answer if it answered your question.

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

subash_sharma — Mon, 12 Jun 2017 16:19:22 GMT

hey @Sharmadha Sainath, How to increase the limit of tables, I am getting only 100 tables using this script.

I tried using "limit" but limit is not working with 1000, It works when i am limiting the number of tables by giving values below 100.

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

ssainath — Mon, 12 Jun 2017 16:29:21 GMT

@subash sharma , add limit parameter like :

/api/atlas/discovery/search/dsl?query=hive_table+where+db.name%3D%22default%22&limit=500

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

ssainath — Mon, 12 Jun 2017 17:01:01 GMT

@subash sharma , limit parameter should work . How many tables are there in the database you are querying?

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

subash_sharma — Mon, 12 Jun 2017 17:10:00 GMT

Hey @Sharmadha Sainath,

I was not able to add comment so adding my reply here, actually i was using space instead of "=" in limit. It is working now. Thanks

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

ssubramanian — Tue, 13 Jun 2017 04:28:20 GMT

The following gremlin query can also be used the get the count of entity type:

http://localhost:21000/api/atlas/discovery/search/gremlin?query=g.V("__typeName", "hive_db").count()
http://localhost:21000/api/atlas/discovery/search/gremlin?query=g.V("__typeName", "hive_table").count()

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

subash_sharma — Tue, 13 Jun 2017 21:33:58 GMT

Thank you @Sarath Subramanian. Right now i am using DSL for searching the tables in a schema. Let me know which one should be good in terms of performance. Gremlin or DSL.

Re: How to fetch total numbers of tables available in a Schema using Rest API search techniques ?

ssainath — Wed, 14 Jun 2017 16:17:36 GMT

@subash sharma , DSL query makes GREMLIN call internally to fetch results.