Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Atlas: bulk tagging multiple guid's ends up in ATLAS-500-00-007

Highlighted

Atlas: bulk tagging multiple guid's ends up in ATLAS-500-00-007

Contributor

This looks like a question I've asked earlier, but that was about bulk creating entities. That gave a similar error, but it worked. Now I'm making a Python script that can read a CSV file with HDFS paths and information classifications from which I create a JSON to use in the Atlas REST API endpoint v2/entity/bulk/classification. I've used this endpoint before, but not with multiple guid's.

Here is the JSON file I've generated:

{
  "classification" : {
    "typeName" : "Internal",
    "attributes" : {}
  },
  "entityGuids" : [ "d338c66b-eb0c-407e-9b03-c4b8f7115dd4", "050e089c-37df-4490-9518-1ed887abcaeb", "de97606a-b902-488e-8071-fdc29d23e9e3" ]
}

This the curl command:

curl -X POST -d @atlas_hdfs_path_classification2.json -u username -H 'Content-Type: application/json;  charset=UTF-8' http://sandbox.hortonworks.com:21000/api/atlas/v2/entity/bulk/classification

It actually does tag one guid, but only the first one in the list (d338c66b-eb0c-407e-9b03-c4b8f7115dd4). Then it ends with this error:

{"errorCode":"ATLAS-500-00-007","errorMessage":"Failed to notify for change {0}"}

I really would like this to work, otherwise I have to generate a whole bunch of single classification commands in a shell script or something.

3 REPLIES 3
Highlighted

Re: Atlas: bulk tagging multiple guid's ends up in ATLAS-500-00-007

Contributor

I'm now doing API calls per guid, to get around this issue.

Highlighted

Re: Atlas: bulk tagging multiple guid's ends up in ATLAS-500-00-007

Contributor

I have tested it my side on 2.6.4 sandbox and all 3 were attached for me. (You don't need -X POST together with -d.)

curl -d @atlas_hdfs_path_classification2.json -u admin:admin -H 'Content-Type: application/json;  charset=UTF-8' http://127.0.0.1:21000/api/atlas/v2/entity/bulk/classification
{
  "classification": {
    "typeName": "environment",
    "attributes": {}
  },
  "entityGuids": [
    "bb69126d-1e7f-4fef-8e7c-21935ee86bf8",
    "c59e438c-5800-495b-bcc2-c9fe543044a6",
    "9dfb47d3-01fb-4b25-9833-75c9c36d2a40"
  ]
}

On a similar note, I wanted to create multiple classifications in one go. I found this worked, where fileToUpload.dat contained a list of json files.

<fileToUpload.dat xargs -I %  curl -X POST -T "{%}" -u admin:admin -H "Content-Type: application/json" http://127.0.0.1:21000/api/atlas/v2/types/typedefs

But the below only created the first classification. No idea why!

cat pii.json pii_attribute.json | curl -d@- -u admin:admin -H "Content-Type: application/json" http://127.0.0.1:21000/api/atlas/v2/types/typedefs
Highlighted

Re: Atlas: bulk tagging multiple guid's ends up in ATLAS-500-00-007

Contributor

Thanks @Laura Ngo .

I still got the same issue that only one GUID is classified on my HDP 2.6.3 sandbox. But I've noticed that several errors that I get on my sandbox, don't happen on the HDP 2.6.3 cluster at my customer.

About that JSON list: the Atlas REST API works in mysterious ways sometimes.

Don't have an account?
Coming from Hortonworks? Activate your account here