Member since
03-29-2016
36
Posts
12
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1524 | 04-08-2018 09:30 PM | |
866 | 03-15-2018 08:13 PM | |
1271 | 03-14-2018 04:10 PM | |
2225 | 03-14-2018 03:48 PM | |
1509 | 02-19-2018 01:03 AM |
08-30-2019
03:27 PM
can we use the same on impala or hbase tables
... View more
04-08-2018
09:30 PM
1 Kudo
It's not elegant, but this custom masking seems to work. cast(mask(cast(to_date({col}) as date), 'x', 'x', 'x', -1, '1', 1, 0, -1) as timestamp)
... View more
03-16-2018
03:38 AM
@Laura Ngo - the API shared by @Madhan Neethiraj is the correct one which you can use. However, in addition to it, I would like to inform you that if you have more than 10000 results and want to scroll between them you can use offset option like below query. curl -k -u admim:admin -H "Content-type:application/json"-X GET https://url:port/api/atlas/v2/earch/dsl?limit=10000&offset=20000&query=hive_column%20where%20__state%3D%27ACTIVE%27%20and%20qualifiedName%20like%20%27prod_%2A_data_lake%2A%27%20selct%20qualifiedName%2Cname%2C__guid | python -m json.tool > hive_column_prod_data_lake.json
... View more
03-05-2018
08:44 PM
You are exactly right - thank you. Both the Atlas tag and the Ranger policy were in caps but I don't think Ranger Audit likes caps. I changed both to lower and the access is denied. access-denied.png Thanks so much for your help. (I've never been so happy to see an 'access denied' message!)
... View more
02-19-2018
01:31 PM
@Laura Ngo Once you have the super tag "security_protection"created , following can be POSTed to API to create Types REST v2 API sub classifications. Note that the superTypes list contains the security_protection classification. http://localhost:21000/api/atlas/v2/types/typedefs?type=classification {
"classificationDefs":[
{
"name":"disk_encryption",
"description":"",
"superTypes":["security_protection" ],
"attributeDefs":[]
},
{
"name":"field_encryption",
"description":"",
"superTypes":["security_protection" ],
"attributeDefs":[]
}
]
}
... View more
06-28-2017
08:17 AM
I'm posting this in case anyone finds it useful. There's now a way for metamodels to inherit values from other values, but in 0.8 you can use the qualifiedName instead of the guid, which is much better. Type POST http://127.0.0.1:21000/api/atlas/v2/types/typedefs {
"enumDefs":[],
"structDefs":[],
"classificationDefs":[],
"entityDefs":[
{
"superTypes":[
"DataSet"
],
"name":"test_entity_18",
"description":"test_entity_18",
"attributeDefs":[
{
"name":"test_18",
"isOptional":true,
"isUnique":true,
"isIndexable":false,
"typeName":"string",
"valuesMaxCount":1,
"cardinality":"SINGLE",
"valuesMinCount":0
},
{
"name":"test_18_db",
"isOptional":true,
"isUnique":true,
"isIndexable":false,
"typeName":"hive_db",
"valuesMaxCount":1,
"cardinality":"SINGLE",
"valuesMinCount":0
}
]
}
]
} Entity POST http://127.0.0.1:21000/api/atlas/v2/entity {
"entity": {
"typeName": "test_entity_18",
"createdBy": "admin",
"updatedBy": "admin",
"attributes": {
"description": "test decription",
"name": "test_entity_18",
"owner": "admin",
"qualifiedName": "test_entity_18",
"test_18": "attr1",
"test_18_db": {
"typeName": "hive_db",
"uniqueAttributes": {
"qualifiedName": "default@Sandbox"
}
}
},
"guid": -1
},
"referredEntities": {}
} I learnt this from here - https://issues.apache.org/jira/browse/ATLAS-1506
... View more
06-19-2017
05:27 PM
3 Kudos
@Laura Ngo, "options" was added to AtlasBaseTypeDef to allow additional metadata properties to be included with the type definition. These properties allow users to specify any key/value property or constraint on the type definition e.g. cluster = test_cluster contains_all_primitive_attributes=true The options map is not being used currently but added for future use cases.
... View more
06-12-2017
07:38 PM
4 Kudos
Hi @Laura Ngo, The documentation for 'Atlas types' is still not updated with v2 types. It should ideally be - "Composite metatypes – Entity, Struct and Classification" Struct Type: You cannot create instances for Struct types as they do not have an identity of their own (it doesn't have a GUID), it always exists with an entity. e.g. entity attribute can be of type struct. Classification type: Each classification instance is associated with an entity. The v2 type name for "trait" is "classification" and it is derived from Struct. e.g. Associate tag/classification to an entity. Checkout Working with Atlas Tags to know more about how you can assign tag/classification to an entity.
... View more
03-14-2017
07:58 AM
Yes, Atlas 0.8 will have all the new v2 APIs for types, entities, lineage and discovery. The corresponding HDP release will be 2.6
... View more
06-17-2016
06:48 PM
2 Kudos
@Laura Ngo I ran your type definition through the REST API for Atlas 0.6. It is valid and does show up as a valid tag in the Atlas UI. From the AtlasUI, I added the tag to a hive table entity successfully and then gave the tag some attributes. It does not look like the UI is able to show or edit the array type attributes for the tags but I was able to call the REST API (the GUID on the end of the URI is the entity id): curl -X GET http://localhost:21000/api/atlas/entities/a6f3e6c8-57f6-45ce-98e7-ea14a1f29211 and get the following result: {
"requestId": "qtp1635546341-110 - 8bb20991-84e5-4dc5-a678-3dbe81cb52a2",
"GUID": "a6f3e6c8-57f6-45ce-98e7-ea14a1f29211",
"definition": {
"jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
"jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id": "a6f3e6c8-57f6-45ce-98e7-ea14a1f29211",
"version": 0,
"typeName": "hive_column"
},
"typeName": "hive_column",
"values": {
"comment": null,
"qualifiedName": "hr.employee.location@erietp",
"type": "string",
"name": "location"
},
"traitNames": [
"PII",
"api_test_set"
],
"traits": {
"PII": {
"jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
"typeName": "PII",
"values": {
}
},
"api_test_set": {
"jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
"typeName": "api_test_set",
"values": {
"collection_test": [
"test"
],
"set_test": [
"test"
]
}
}
}
}
} Notice that the collection_test and set_test values show up as arrays denoted by [ ] and they are populated. As stated before, this will not show up in the UI but the value are persisted. I have not tried to add more elements to the arrays within the trait through the REST API but I don't see any reason why that would not work. If you were planning to use the value within the tags for some custom purpose you should be good to go.
... View more