Atlas REST API search of a table fails

[#1]

I want to allow users that store data on our HDP 2.6.3 cluster, to tag to their data in Atlas with the push a button. So I'm preparing the REST API commands for that. And because Atlas is not implemented here, I'm trying this on a HDP 2.6.3 Sandbox on a VirtualBox.

Atlas is running and the REST API works. For example, I can run this:

curl -iv -X GET http://sandbox.hortonworks.com:21000/api/atlas/entities?type=hive_table

And this results in:

{
"requestId": "pool-2-thread-4 - d7ab6dc3-2464-482d-b78b-24f5e14bfd49",
"typeName": "hive_table",
"results": [
"faedff26-819c-47ee-9cdf-77b4ba8bc547",
"dfbe373d-b672-418e-8f45-5e285b64dd7d",
"f8861c18-6ba2-455d-9024-96abf01387f1",
"9503d0e1-37d7-4456-8b79-30dda3199f67",
"c6fc7997-648a-4154-9bca-b265f4882ae5",
"155265ec-dbfe-4951-b051-2b09c47a6c7f",
"908cba22-cfa3-429e-a116-51dcdcbfb003",
"8de87c74-3c5d-4b5c-8218-316f2b785f5d",
"c87f4c84-54e7-4c07-ac77-f32811b40fa5",
"aaabeeb3-7528-4f76-8f65-443d26bb266e",
"59234a6b-7d3f-415f-b430-aae2074a0bb0",
"83cd29e8-cca9-4595-9d3b-8479cd2fc3ee",
"70f945be-7c3a-4580-86ac-695b345d0d7f"
],
"count": 13
}

Now I want to search for a Hive table and I just can't seem to get that working. Just to prove the table is there, here is a screenshot from the Atlas GUI:

atlas-search-advanced-hive-table.png(30.6 kB)

I've tried many variations of the DSL search. None of them work. These are just a couple of the many variations that I've tried:

curl -iv -u holger_gov:holger_gov -X GET http://sandbox.hortonworks.com:21000/api/atlas/discovery/search/dsl?query=hive_table+where+name='asteroids'

Message: {"error":"Invalid expression : hive_table where name=asteroids"}

curl -iv -u holger_gov:holger_gov -X GET http://sandbox.hortonworks.com:21000/api/atlas/v2/search/dsl?query=hive_table+where+name='asteroids'

Message: There was an error processing your request. It has been logged (ID 7da439947f22992b).

curl -iv -u holger_gov:holger_gov -X GET --header 'Accept: text/html' 'http://sandbox.hortonworks.com:21000/v2/search/dsl?query=name%3D'asteroids''

Message:

<body><h2>HTTP ERROR 404</h2><p>Problem accessing /v2/search/dsl. Reason:<pre>  Not Found</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/></body>

curl -iv -u holger_gov:holger_gov -X GET http://sandbox.hortonworks.com:21000/api/atlas/discovery/search/dsl?query=hive_table+where+name='asteroids'

Message: {"error":"Invalid expression : hive_table where name=asteroids"}

curl -iv -u holger_gov:holger_gov -X GET http://sandbox.hortonworks.com:21000/api/atlas/discovery/search/dsl?query=hive_table+where%20name%3D'asteroids'

Result: {"error":"Invalid expression : hive_table where name=asteroids"}

What am I missing here?

[#2]

@Marcel-Jan Krijgsman

Can you please try this once

curl -iv -u holger_gov:holger_gov -X GET http://sandbox.hortonworks.com:21000/api/atlas/v2/search/dsl?query=where%20name%3D%asteroids%27&type...

Thanks,

Aditya

[#3]

The message I get is:

{"errorCode":"ATLAS-500-00-004","errorMessage":"Discovery query failed where name=%asteroids'"}

[#4]

@Marcel-Jan Krijgsman,

This should work

curl -iv -u holger_gov:holger_gov -X GET http://sandbox.hortonworks.com:21000/api/atlas/v2/search/dsl?typeName=hive_table&query=where+name%3D...

Thanks,

Aditya

[#5]

Hey yes! That already looks a lot more like a query response. It's not just the asteroids hive_table though, but it's getting somewhere.

{
  "queryType": "DSL",
  "queryText": "`hive_table` ",
  "entities": [
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "spark",
  "qualifiedName": "asteroids.asteroids_raw@Sandbox",
  "name": "asteroids_raw",
  "description": null
  },
  "guid": "faedff26-819c-47ee-9cdf-77b4ba8bc547",
  "status": "ACTIVE",
  "displayText": "asteroids_raw",
  "classificationNames": [
  "Applicatie_X"
  ]
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "spark",
  "qualifiedName": "asteroids.asteroids@Sandbox",
  "name": "asteroids",
  "description": null
  },
  "guid": "dfbe373d-b672-418e-8f45-5e285b64dd7d",
  "status": "ACTIVE",
  "displayText": "asteroids",
  "classificationNames": []
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "hive",
  "qualifiedName": "xademo.call_detail_records@Sandbox",
  "name": "call_detail_records",
  "description": null
  },
  "guid": "70f945be-7c3a-4580-86ac-695b345d0d7f",
  "status": "ACTIVE",
  "displayText": "call_detail_records",
  "classificationNames": []
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "hive",
  "qualifiedName": "xademo.recharge_details@Sandbox",
  "name": "recharge_details",
  "description": null
  },
  "guid": "f8861c18-6ba2-455d-9024-96abf01387f1",
  "status": "ACTIVE",
  "displayText": "recharge_details",
  "classificationNames": []
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "hive",
  "qualifiedName": "xademo.customer_details@Sandbox",
  "name": "customer_details",
  "description": null
  },
  "guid": "155265ec-dbfe-4951-b051-2b09c47a6c7f",
  "status": "ACTIVE",
  "displayText": "customer_details",
  "classificationNames": []
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "hive",
  "qualifiedName": "foodmart.inventory_fact_1998@Sandbox",
  "name": "inventory_fact_1998",
  "description": null
  },
  "guid": "aaabeeb3-7528-4f76-8f65-443d26bb266e",
  "status": "ACTIVE",
  "displayText": "inventory_fact_1998",
  "classificationNames": []
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "hive",
  "qualifiedName": "foodmart.sales_fact_dec_1998@Sandbox",
  "name": "sales_fact_dec_1998",
  "description": null
  },
  "guid": "59234a6b-7d3f-415f-b430-aae2074a0bb0",
  "status": "ACTIVE",
  "displayText": "sales_fact_dec_1998",
  "classificationNames": []
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "hive",
  "qualifiedName": "foodmart.product@Sandbox",
  "name": "product",
  "description": null
  },
  "guid": "c6fc7997-648a-4154-9bca-b265f4882ae5",
  "status": "ACTIVE",
  "displayText": "product",
  "classificationNames": []
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "hive",
  "qualifiedName": "foodmart.customer@Sandbox",
  "name": "customer",
  "description": null
  },
  "guid": "9503d0e1-37d7-4456-8b79-30dda3199f67",
  "status": "ACTIVE",
  "displayText": "customer",
  "classificationNames": []
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "hive",
  "qualifiedName": "foodmart.store@Sandbox",
  "name": "store",
  "description": null
  },
  "guid": "83cd29e8-cca9-4595-9d3b-8479cd2fc3ee",
  "status": "ACTIVE",
  "displayText": "store",
  "classificationNames": []
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "hive",
  "qualifiedName": "default.sample_07@Sandbox",
  "name": "sample_07",
  "description": null
  },
  "guid": "908cba22-cfa3-429e-a116-51dcdcbfb003",
  "status": "ACTIVE",
  "displayText": "sample_07",
  "classificationNames": []
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "hive",
  "qualifiedName": "default.sample*_08@Sandbox",
  "name": "sample_08",
  "description": null
  },
  "guid": "c87f4c84-54e7-4c07-ac77-f32811b40fa5",
  "status": "ACTIVE",
  "displayText": "sample_08",
  "classificationNames": []
  },
  {
  "typeName": "hive_table",
  "attributes": {
  "owner": "raj_ops",
  "qualifiedName": "default.employee@Sandbox",
  "name": "employee",
  "description": null
  },
  "guid": "8de87c74-3c5d-4b5c-8218-316f2b785f5d",
  "status": "ACTIVE",
  "displayText": "employee",
  "classificationNames": []
  }
  ]
}

[#6]

@Marcel-Jan Krijgsman,

Glad that it worked. Can you please accept the answer. This will be really helpful for other community users for finding the answer quickly.

[#7]

If this is the best possible result I certainly will accept the answer. Because it's something I can work with.

But I really would like a Atlas REST API call that just gives results for the asteroids table only. That should be possible, shouldn't it?

Because now this command:

curl -iv -u holger_gov:holger_gov -X GET http://sandbox.hortonworks.com:21000/api/atlas/v2/search/dsl?typeName=hive_table&query=where+name%3D...

Gives the same result as if we omit the "where name='asteroids'" part:

curl -iv -u holger_gov:holger_gov -X GET http://sandbox.hortonworks.com:21000/api/atlas/v2/search/dsl?typeName=hive_table

[#8]

@Marcel-Jan Krijgsman,

I tried this and it returns only the queried table

curl -X GET \
  'http://sandbox.hortonworks.com:21000/api/atlas/v2/search/dsl?typeName=hive_table&query=where%20name%3D%22asteroids%22' \
  -H 'authorization: Basic YWRtaW46YWRtaW4='

[#9]

@Marcel-Jan Krijgsman

Glad that it worked. Can you please accept the answer. This will be really helpful for other community users for finding the answer quickly.

[#10]

That's it! Great work. Thank you very much.