Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Populating ATLAS with BULK API

Solved Go to solution

Populating ATLAS with BULK API

New Contributor

Hi,

I'm kinda a beginner in using REST API and i want to create multiple entities at the same time.
Watching documentation, you can do it with v2/ENTITY/BULK, however i don't understand how the json file have to be formatted.


When I create ONE entity, I do it with v2/ENTITY i just do it like this:

curl -s -i -X POST -H 'Content-Type: application/json' 'Accept: application/json' -u admin:$MDP_ATLAS "$ATLAS_BASE_URL/v2/entity" -d '{"entities":{ "typeName": "Referenceable", "attributes": {"qualifiedName":"some.qualified.name1"}}}'

 

But with BULK the doc is saying to format the file like this:

{
  "entities" : [ {
    "classifications" : [ {
      "entityGuid" : "...",
      "entityStatus" : "ACTIVE",
      "propagate" : true,
      "removePropagationsOnEntityDelete" : true,
      "validityPeriods" : [ { }, { } ],
      "attributes" : {
        "property1" : { },
        "property2" : { }
      },
      "typeName" : "..."
    }, {
      "entityGuid" : "...",
      "entityStatus" : "ACTIVE",
      "propagate" : true,
      "removePropagationsOnEntityDelete" : true,
      "validityPeriods" : [ { }, { } ],
      "attributes" : {
        "property1" : { },
        "property2" : { }
      },
      "typeName" : "..."
    } ],
    "createTime" : 12345,
    "createdBy" : "...",
    "guid" : "...",
    "homeId" : "...",
    "meanings" : [ {
      "confidence" : 12345,
      "createdBy" : "...",
      "description" : "...",
      "displayText" : "...",
      "expression" : "...",
      "relationGuid" : "...",
      "source" : "...",
      "status" : "OBSOLETE",
      "steward" : "...",
      "termGuid" : "..."
    }, {
      "confidence" : 12345,
      "createdBy" : "...",
      "description" : "...",
      "displayText" : "...",
      "expression" : "...",
      "relationGuid" : "...",
      "source" : "...",
      "status" : "PROPOSED",
      "steward" : "...",
      "termGuid" : "..."
    } ],
    "provenanceType" : 12345,
    "proxy" : true,
    "relationshipAttributes" : {
      "property1" : { },
      "property2" : { }
    },
    "status" : "ACTIVE",
    "updateTime" : 12345,
    "updatedBy" : "...",
    "version" : 12345,
    "attributes" : {
      "property1" : { },
      "property2" : { }
    },
    "typeName" : "..."
  }, {
    "classifications" : [ {
      "entityGuid" : "...",
      "entityStatus" : "ACTIVE",
      "propagate" : true,
      "removePropagationsOnEntityDelete" : true,
      "validityPeriods" : [ { }, { } ],
      "attributes" : {
        "property1" : { },
        "property2" : { }
      },
      "typeName" : "..."
    }, {
      "entityGuid" : "...",
      "entityStatus" : "ACTIVE",
      "propagate" : true,
      "removePropagationsOnEntityDelete" : true,
      "validityPeriods" : [ { }, { } ],
      "attributes" : {
        "property1" : { },
        "property2" : { }
      },
      "typeName" : "..."
    } ],
    "createTime" : 12345,
    "createdBy" : "...",
    "guid" : "...",
    "homeId" : "...",
    "meanings" : [ {
      "confidence" : 12345,
      "createdBy" : "...",
      "description" : "...",
      "displayText" : "...",
      "expression" : "...",
      "relationGuid" : "...",
      "source" : "...",
      "status" : "DISCOVERED",
      "steward" : "...",
      "termGuid" : "..."
    }, {
      "confidence" : 12345,
      "createdBy" : "...",
      "description" : "...",
      "displayText" : "...",
      "expression" : "...",
      "relationGuid" : "...",
      "source" : "...",
      "status" : "VALIDATED",
      "steward" : "...",
      "termGuid" : "..."
    } ],
    "provenanceType" : 12345,
    "proxy" : true,
    "relationshipAttributes" : {
      "property1" : { },
      "property2" : { }
    },
    "status" : "DELETED",
    "updateTime" : 12345,
    "updatedBy" : "...",
    "version" : 12345,
    "attributes" : {
      "property1" : { },
      "property2" : { }
    },
    "typeName" : "..."
  } ],
  "referredEntities" : {
    "property1" : {
      "classifications" : [ {
        "entityGuid" : "...",
        "entityStatus" : "DELETED",
        "propagate" : true,
        "removePropagationsOnEntityDelete" : true,
        "validityPeriods" : [ { }, { } ],
        "attributes" : {
          "property1" : { },
          "property2" : { }
        },
        "typeName" : "..."
      }, {
        "entityGuid" : "...",
        "entityStatus" : "DELETED",
        "propagate" : true,
        "removePropagationsOnEntityDelete" : true,
        "validityPeriods" : [ { }, { } ],
        "attributes" : {
          "property1" : { },
          "property2" : { }
        },
        "typeName" : "..."
      } ],
      "createTime" : 12345,
      "createdBy" : "...",
      "guid" : "...",
      "homeId" : "...",
      "meanings" : [ {
        "confidence" : 12345,
        "createdBy" : "...",
        "description" : "...",
        "displayText" : "...",
        "expression" : "...",
        "relationGuid" : "...",
        "source" : "...",
        "status" : "IMPORTED",
        "steward" : "...",
        "termGuid" : "..."
      }, {
        "confidence" : 12345,
        "createdBy" : "...",
        "description" : "...",
        "displayText" : "...",
        "expression" : "...",
        "relationGuid" : "...",
        "source" : "...",
        "status" : "OTHER",
        "steward" : "...",
        "termGuid" : "..."
      } ],
      "provenanceType" : 12345,
      "proxy" : true,
      "relationshipAttributes" : {
        "property1" : { },
        "property2" : { }
      },
      "status" : "ACTIVE",
      "updateTime" : 12345,
      "updatedBy" : "...",
      "version" : 12345,
      "attributes" : {
        "property1" : { },
        "property2" : { }
      },
      "typeName" : "..."
    },
    "property2" : {
      "classifications" : [ {
        "entityGuid" : "...",
        "entityStatus" : "ACTIVE",
        "propagate" : true,
        "removePropagationsOnEntityDelete" : true,
        "validityPeriods" : [ { }, { } ],
        "attributes" : {
          "property1" : { },
          "property2" : { }
        },
        "typeName" : "..."
      }, {
        "entityGuid" : "...",
        "entityStatus" : "ACTIVE",
        "propagate" : true,
        "removePropagationsOnEntityDelete" : true,
        "validityPeriods" : [ { }, { } ],
        "attributes" : {
          "property1" : { },
          "property2" : { }
        },
        "typeName" : "..."
      } ],
      "createTime" : 12345,
      "createdBy" : "...",
      "guid" : "...",
      "homeId" : "...",
      "meanings" : [ {
        "confidence" : 12345,
        "createdBy" : "...",
        "description" : "...",
        "displayText" : "...",
        "expression" : "...",
        "relationGuid" : "...",
        "source" : "...",
        "status" : "PROPOSED",
        "steward" : "...",
        "termGuid" : "..."
      }, {
        "confidence" : 12345,
        "createdBy" : "...",
        "description" : "...",
        "displayText" : "...",
        "expression" : "...",
        "relationGuid" : "...",
        "source" : "...",
        "status" : "DISCOVERED",
        "steward" : "...",
        "termGuid" : "..."
      } ],
      "provenanceType" : 12345,
      "proxy" : true,
      "relationshipAttributes" : {
        "property1" : { },
        "property2" : { }
      },
      "status" : "DELETED",
      "updateTime" : 12345,
      "updatedBy" : "...",
      "version" : 12345,
      "attributes" : {
        "property1" : { },
        "property2" : { }
      },
      "typeName" : "..."
    }
  }
}

 

And I don't understand how I should write that file.

Let's say I want to create two entities with BULK:

{"entities":{ "typeName": "Referenceable", "attributes": {"qualifiedName":"some.qualified.name1"}}}
AND
{"entities":{ "typeName": "Referenceable", "attributes": {"qualifiedName":"some.qualified.name2"}}}

 

How can I do that ?

Can someone explain me how to write that file ?


I thank you for the time you took to read my request, have a good day .

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Populating ATLAS with BULK API

New Contributor

**bleep** I'm so dumb

curl -s -i -X POST -H 'Content-Type: application/json' 'Accept: application/json' -u admin:$MDP_ATLAS "$ATLAS_BASE_URL/v2/entity/bulk" -d '{"entities":[{ "typeName": "Referenceable", "attributes": {"qualifiedName":"some.qualified.name1"}}, { "typeName": "Referenceable", "attributes": {"qualifiedName":"some.qualified.name2"}}]}'

 

I tried it before but without putting entites in [ ]  

View solution in original post

1 REPLY 1
Highlighted

Re: Populating ATLAS with BULK API

New Contributor

**bleep** I'm so dumb

curl -s -i -X POST -H 'Content-Type: application/json' 'Accept: application/json' -u admin:$MDP_ATLAS "$ATLAS_BASE_URL/v2/entity/bulk" -d '{"entities":[{ "typeName": "Referenceable", "attributes": {"qualifiedName":"some.qualified.name1"}}, { "typeName": "Referenceable", "attributes": {"qualifiedName":"some.qualified.name2"}}]}'

 

I tried it before but without putting entites in [ ]  

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here