Created 07-10-2017 12:46 PM
Hi,
Can anyone please help me in understanding SELECT clause in DSL? Following mentioned query returns no rows though there is an entity named as "Reporting" of type "DB" having attributes as name and owner.
DB where name="Reporting" select name, owner
Created 07-10-2017 01:05 PM
The select lets you "select" a particular attribute of an entity
DB where name="Reporting" select name, owner
displays the name and owner of db whose name is "Reporting"
Currently UI doesn't display results but REST GET API call actually returns the result.
Example :
DSL query : hive_table name = "table1" select qualifiedName,owner
response :
{queryType: "DSL", queryText: "`hive_table` name="table1" select qualifiedName,owner", attributes: {name: ["qualifiedName", "owner" ], values: [ ["default.table1@cl1", "atlas" ] ] } }
Created 07-10-2017 01:05 PM
The select lets you "select" a particular attribute of an entity
DB where name="Reporting" select name, owner
displays the name and owner of db whose name is "Reporting"
Currently UI doesn't display results but REST GET API call actually returns the result.
Example :
DSL query : hive_table name = "table1" select qualifiedName,owner
response :
{queryType: "DSL", queryText: "`hive_table` name="table1" select qualifiedName,owner", attributes: {name: ["qualifiedName", "owner" ], values: [ ["default.table1@cl1", "atlas" ] ] } }
Created 07-11-2017 06:51 AM
@Sharmadha Sainath Could you please also guide me how is it using groupby and other functions liek max, min with some example? Does their result reflects on UI or not? Also, what features e.g wildcards (? . * etc) are available in full-text provided by Atlas? and how can we use them (examples)?
Created on 07-11-2017 07:22 AM - edited 08-17-2019 05:18 PM
Queries like this are also not returning anything e.g using groupby with select
Created 07-11-2017 07:27 AM
@Sharmadha Sainath could you please also tell me how does this works
`Log Data` ??? what am I supposed to write in single quotes and how do you explain the results?
Created 07-11-2017 08:14 AM
@Saba Baig, you can search for any entities tagged with a particular tag name using the single quotes. for e.g. `Log Data` searches for all entities in atlas tagged as Log Data. Only classification name is supported in single quotes.
Created 07-11-2017 06:40 AM
@Sharmadha Sainath can you please guide me on this? as the method "searchUsingDslQuery" from EntityDiscoveryService.java is getting the result of the fetched attributes but then it only adds the attributes to the object of AtlasSearchResult...neither an AtlasVertex nor contains theInstance or theTrait.... Could you please guide me on this?
Created 07-11-2017 08:20 AM
There are few Apache JIRAs filed for the same :
https://issues.apache.org/jira/browse/ATLAS-1874
https://issues.apache.org/jira/browse/ATLAS-1639
However V1 APIs work perfectly .
For example : Query : hive_table groupby(owner) select owner,count() URL encoded : http://localhost:21000/api/atlas/discovery/search/dsl?limit=25&query=hive_table+groupby(owner)+selec... Response : {requestId: "pool-2-thread-9 - 8a45b7a6-0cd9-4e9d-8b71-66e313f5abf1", query: "hive_table groupby(owner) select owner,count()", queryType: "dsl", count: 2, results: [ {$typeName$: "__tempQueryResultStruct74", count(): 9, owner: "hrt_qa" }, {$typeName$: "__tempQueryResultStruct74", count(): 2, owner: "anonymous" } ], dataType: {typeName: "__tempQueryResultStruct74", typeDescription: null, typeVersion: "1.0", attributeDefinitions: [ {name: "owner", dataTypeName: "string", multiplicity: {lower: 0, upper: 1, isUnique: false }, isComposite: false, isUnique: false, isIndexable: false, reverseAttributeName: null }, {name: "count()", dataTypeName: "long", multiplicity: {lower: 0, upper: 1, isUnique: false }, isComposite: false, isUnique: false, isIndexable: false, reverseAttributeName: null } ] } }
This query groups the owner of hive_table and and lists the owner ,count in each group . The concept is similar to that of SQL.
Since the UI currently uses V2 query , you may not see the results on UI. But you can always rely on REST APIs.
For this particular kind of queries , please use V1 APIs for now .
Created 07-11-2017 08:30 AM
>> Also, what features e.g wildcards (? . * etc) are available in full-text provided by Atlas? and how can we use them (examples)?
Atlas uses lucene query syntax for its full text search (basic search).
Please refer lucene query usage here :
Example:
If I have 3 hive_table entities in Atlas , table_a , table_b , table_ab
Query : 1) type = hive_table , query = table_? http://localhost:21000/api/atlas/v2/search/basic?limit=25&excludeDeletedEntities=true&query=table_?&... Returns 2 entities (table_a , table_b) 2) type = hive_table , query = table_* http://localhost:21000/api/atlas/v2/search/basic?limit=25&excludeDeletedEntities=true&query=table_*&... Returns 3 entities (table_a , table_b , table_ab)
I hope this example makes it clear ...
Note : Wild card search can be done on UI itself.
Created on 07-11-2017 10:31 AM - edited 08-17-2019 05:18 PM
I have an entity of type "Table" named as "sales_fact". I tried to get that using api
http://localhost:21000/api/atlas/v2/search/basic?limit=25&includeDeletedEntities=true&query=sales_fa...
=====Response ===== { "queryType": "BASIC", "type": "Table", "entities": [ { "typeName": "Table", "attributes": { "owner": "Joe", "qualifiedName": "sales_fact", "name": "sales_fact", "description": "sales fact table" }, "guid": "c9838518-041f-4759-b3e5-01147b1526d1", "status": "ACTIVE", "displayText": "sales_fact", "classificationNames": [ "Fact" ] } ]}
But when I tried to the same search (Basic using query) I typed sales_fa?t and I got invalid Expression. I added a breakpoint in the API but it was never called. I think there is some validation applied on that field that omits any character like ? from the query before submitting it to API. Please guide.
I think, this could be the reason
Apache Lucene Analysis Pipeline