Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
smartdatabundle
smartdatabundle
Contributor
My Accepted Solutions
Show All

Re: Filter Option with Ranger for Hive Tables

by Contributor smartdatabundle in Support Questions
‎08-07-2017 07:18 AM
‎08-07-2017 07:18 AM
Hi @Greg Keys Thks so much for your comment. I will try your advices and let you know. ... View more

Filter Option with Ranger for Hive Tables

by Contributor smartdatabundle in Support Questions
‎08-02-2017 01:37 PM
‎08-02-2017 01:37 PM
Hello, When we use a Filter with Ranger on Hive tables: is it possible to define a dynamic value ? My use case : I have different groups on my application. Each group has at least one user. Data could be accessed by only an appropriate group. My SQL request : select * from table t1, table t2 where t1.id=t2.id and t2.group="group_admin". Instead of "group_admin" I would like to put a dynamic value. Is it possible or any suggestions ? Thks for your helps ... View more

Re: Atlas : how to secure Kafka ?

by Contributor smartdatabundle in Support Questions
‎08-01-2017 12:22 PM
‎08-01-2017 12:22 PM
@Geoffrey Shelton Okot thks for this explanation. ... View more

Re: How To Tuning Apache Atlas Performance?

by Contributor smartdatabundle in Support Questions
‎08-01-2017 09:44 AM
‎08-01-2017 09:44 AM
You could use Messaging API with Kafka to access to entities. It is very impressive. ... View more

Re: Atlas : how to secure Kafka ?

by Contributor smartdatabundle in Support Questions
‎08-01-2017 09:42 AM
‎08-01-2017 09:42 AM
@Graham Martin Thks for your expanation. I am going to install & to use Kerberos. ... View more

Re: Atlas : how to secure Kafka ?

by Contributor smartdatabundle in Support Questions
‎08-01-2017 08:09 AM
‎08-01-2017 08:09 AM
@Graham Martin Thks for your quick reply. Is there an alternative to Kerberos ? May be Apache Knox + LDAP ? I went to the link: indeed, it explains only the use of Kerberos. ... View more

Atlas : how to secure Kafka ?

by Contributor smartdatabundle in Support Questions
‎08-01-2017 07:08 AM
‎08-01-2017 07:08 AM
Hello, I could access all topics on kafka without authentification. My question : how I could secure access on kafka topics ? Thks. ... View more

Re: Atlas : Lineage only with REST API not with K...

by Contributor smartdatabundle in Support Questions
‎07-28-2017 05:41 AM
‎07-28-2017 05:41 AM
Please look at: https://community.hortonworks.com/questions/115749/messaging-api-with-kafka-lineage.html ... View more

Re: Messaging API with Kafka : lineage

by Contributor smartdatabundle in Support Questions
‎07-28-2017 05:37 AM
‎07-28-2017 05:37 AM
@Sarath Subramanian Excellent! It works. Your are the best 🙂 Thks also to @Ashutosh Mestry for helping. ... View more

Re: Messaging API with Kafka : lineage

by Contributor smartdatabundle in Support Questions
‎07-27-2017 07:35 AM
‎07-27-2017 07:35 AM
@Ashutosh Mestry With REST APIs: it works very well. For my use case I need to use Kafka messaging. ... View more

Re: Messaging API with Kafka : lineage

by Contributor smartdatabundle in Support Questions
‎07-26-2017 07:13 AM
‎07-26-2017 07:13 AM
Hi @Ashutosh Mestry 2 entities exist : 513cd2cd-3139-4baa-a017-16f83ca2b383: { "requestId": "pool-2-thread-9 - 8629abee-a74c-43a4-a969-758fd20704e4", "definition": { "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference", "id": { "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id", "id": "513cd2cd-3139-4baa-a017-16f83ca2b383", "version": 0, "typeName": "hdfs_path", "state": "ACTIVE" }, "typeName": "hdfs_path", "values": { "name": "Data_Kafka2", "path": "/user/data/dept.csv", "posixPermissions": null, "createTime": "1970-01-01T00:00:00.000Z", "description": "Test with Kafka", "isSymlink": false, "extendedAttributes": null, "numberOfReplicas": 0, "qualifiedName": "deptData2", "isFile": false, "fileSize": 0, "owner": "admin", "modifiedTime": "1970-01-01T00:00:00.000Z", "clusterName": null, "group": null }, "traitNames": [ "Catalog.Technical.IT", "Retainable" ], "traits": { "Catalog.Technical.IT": { "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct", "typeName": "Catalog.Technical.IT", "values": { "name": "Catalog.Technical.IT", "acceptable_use": null, "description": "technical test", "available_as_tag": false, "taxonomy.namespace": "atlas.taxonomy" } }, "Retainable": { "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct", "typeName": "Retainable", "values": { "retentionPeriod": 100 } } }, "systemAttributes": { "createdBy": "admin", "modifiedBy": "admin", "createdTime": "2017-06-28T07:32:37.288Z", "modifiedTime": "2017-06-29T12:34:30.304Z" } } } d5c27069-d7b6-4b58-8e61-adbdc4b8d208 : { "requestId": "pool-2-thread-6 - e16f006a-962d-45b5-a2b6-6914ec6ea0ab", "definition": { "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference", "id": { "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id", "id": "d5c27069-d7b6-4b58-8e61-adbdc4b8d208", "version": 0, "typeName": "hdfs_path", "state": "ACTIVE" }, "typeName": "hdfs_path", "values": { "name": "Data_Kafka1", "path": "/user/data/dept.csv", "posixPermissions": null, "createTime": "1970-01-01T00:00:00.000Z", "description": "Test with Kafka", "isSymlink": false, "extendedAttributes": null, "numberOfReplicas": 0, "qualifiedName": "deptData", "isFile": false, "fileSize": 0, "owner": "admin", "modifiedTime": "1970-01-01T00:00:00.000Z", "clusterName": null, "group": null }, "traitNames": [ ], "traits": { }, "systemAttributes": { "createdBy": "admin", "modifiedBy": "admin", "createdTime": "2017-07-21T07:40:37.357Z", "modifiedTime": "2017-07-21T07:52:59.405Z" } } } ... View more

Re: Messaging API with Kafka : lineage

by Contributor smartdatabundle in Support Questions
‎07-26-2017 06:41 AM
‎07-26-2017 06:41 AM
Hi @Sarath Subramanian This is my message: { "message": { "entities": [ { "id": { "id": "-1467290565135246000", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id", "state": "ACTIVE", "typeName": "hdfs_path", "version": 0 }, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference", "traitNames": [], "traits": {}, "typeName": "hdfs_path", "values": { "createTime": "1970-01-01T00:00:00.000Z", "description": "Test with Kafka", "extendedAttributes": null, "isSymlink": false, "name": "Data_Kafka", "numberOfReplicas": 0, "owner": "admin", "path": "/user/data/dept.csv", "posixPermissions": null, "qualifiedName": "deptData" } } ], "type": "ENTITY_CREATE", "user": "admin" }, "version": { "version": "1.0.0" } } ... View more

Messaging API with Kafka : lineage

by Contributor smartdatabundle in Support Questions
‎07-25-2017 01:47 PM
‎07-25-2017 01:47 PM
Hello, I try to create a Process Entity with Kafka : { "message": { "entities": [ { "id": { "id": "-1467290565135246000", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id", "state": "ACTIVE", "typeName": "Process", "version": 0 }, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference", "traitNames": [], "traits": {}, "typeName": "Process", "values": { "qualifiedName": "qn_process_cp", "processor_name": "file_copy_", "name": "n_process_cp", "inputs": [{"guid": "d5c27069-d7b6-4b58-8e61-adbdc4b8d208","typeName": "hdfs_path"}], "outputs": [{"guid": "513cd2cd-3139-4baa-a017-16f83ca2b383","typeName": "hdfs_path"}] } } ], "type": "ENTITY_CREATE", "user": "admin" }, "version": { "version": "1.0.0" } } I have this error message (on Atlas log) : 2017-07-25 13:24:58,645 DEBUG - [NotificationHookConsumer thread-0:] ~ Set property __modifiedBy = "admin" to vertex[id=163872776 type=Process guid=null] (GraphHelper:413) 2017-07-25 13:24:58,645 DEBUG - [NotificationHookConsumer thread-0:] ~ <== createStructVertex(Process) (EntityGraphMapper:188) 2017-07-25 13:24:58,646 DEBUG - [NotificationHookConsumer thread-0:] ~ ==> addProperty(vertex[id=163872776 type=Process guid=null], __superTypeNames, Referenceable) (AtlasGraphUtilsV1:132) 2017-07-25 13:24:58,646 DEBUG - [NotificationHookConsumer thread-0:] ~ ==> addProperty(vertex[id=163872776 type=Process guid=null], __superTypeNames, Asset) (AtlasGraphUtilsV1:132) 2017-07-25 13:24:58,646 DEBUG - [NotificationHookConsumer thread-0:] ~ ==> setProperty(vertex[id=163872776 type=Process guid=null], __guid, f789fb6e-ad0e-4702-b787-e2f5f6a61bdf) (AtlasGraphUtilsV1:141) 2017-07-25 13:24:58,646 DEBUG - [NotificationHookConsumer thread-0:] ~ Setting property __guid in vertex[id=163872776 type=Process guid=null] (AtlasGraphUtilsV1:159) 2017-07-25 13:24:58,648 DEBUG - [NotificationHookConsumer thread-0:] ~ ==> setProperty(vertex[id=163872776 type=Process guid=f789fb6e-ad0e-4702-b787-e2f5f6a61bdf], __version, 0) (AtlasGraphUtilsV1:141) 2017-07-25 13:24:58,648 DEBUG - [NotificationHookConsumer thread-0:] ~ Setting property __version in vertex[id=163872776 type=Process guid=f789fb6e-ad0e-4702-b787-e2f5f6a61bdf] (AtlasGraphUtilsV1:159) 2017-07-25 13:24:58,648 DEBUG - [NotificationHookConsumer thread-0:] ~ ==> mapAttributes(CREATE, Process) (EntityGraphMapper:211) 2017-07-25 13:24:58,648 DEBUG - [NotificationHookConsumer thread-0:] ~ ==> AtlasTypeRegistry.getType(Process) (AtlasStructType:82) 2017-07-25 13:24:58,648 DEBUG - [NotificationHookConsumer thread-0:] ~ <== AtlasTypeRegistry.getType(Process): org.apache.atlas.type.AtlasEntityType@75b239ad (AtlasStructType:108) 2017-07-25 13:24:58,649 DEBUG - [NotificationHookConsumer thread-0:] ~ ==> mapArrayValue(org.apache.atlas.repository.store.graph.v1.AttributeMutationContext@fc69927c) (EntityGraphMapper:550) 2017-07-25 13:24:58,649 DEBUG - [NotificationHookConsumer thread-0:] ~ ==> mapObjectIdValue(org.apache.atlas.repository.store.graph.v1.AttributeMutationContext@fc5b7afb) (EntityGraphMapper:437) 2017-07-25 13:24:58,649 DEBUG - [NotificationHookConsumer thread-0:] ~ <== AtlasErrorCode.getMessage([null]) (AtlasErrorCode:125) 2017-07-25 13:24:58,650 DEBUG - [NotificationHookConsumer thread-0:] ~ ==> AtlasErrorCode.getMessage([null]): ObjectId is not valid null (AtlasErrorCode:132) 2017-07-25 13:24:58,650 WARN - [NotificationHookConsumer thread-0:] ~ Error handling message (NotificationHookConsumer:325) org.apache.atlas.exception.AtlasBaseException: ObjectId is not valid null at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapObjectIdValue(EntityGraphMapper.java:455) at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapCollectionElementsToVertex(EntityGraphMapper.java:644) at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapArrayValue(EntityGraphMapper.java:568) at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapToVertexByTypeCategory(EntityGraphMapper.java:318) at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttribute(EntityGraphMapper.java:260) at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttributes(EntityGraphMapper.java:221) at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttributesAndClassifications(EntityGraphMapper.java:127) at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.createOrUpdate(AtlasEntityStoreV1.java:230) at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.createOrUpdate(AtlasEntityStoreV1.java:246) at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:266) at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:235) Atlas version : 0.8 Kafka version : 0.10.1 Any suggestions ? Thks ... View more

Re: How to create a New Entity in Apache Atlas 0.7...

by Contributor smartdatabundle in Support Questions
‎06-28-2017 01:21 PM
‎06-28-2017 01:21 PM
I never did it but I could suggest : Create a new simple type : curl -u admin:admin -ik -H "Content-Type: application/json" -X POST -d '{"enumDefs" : [], "structDefs" : [], "classificationDefs": [], "entityDefs" : [{"name": "nifi_process_test", "superTypes": ["DataSet"], "typeVersion": "1.0", "attributeDefs": [{"name": "processor_name", "typeName": "string", "cardinality": "SINGLE", "isIndexable": false, "isOptional": false, "isUnique": false } ] }] }' http://YOUR_HOST:21000/api/atlas/v2/types/typedefs I know that Nifi could connect to KAFKA via a processor. So, you just have to publish into Kafka Topic like this : 1- Open Producer console: $KAFKA_HOME/kafka-console-producer.sh --broker-list YOUR_HOST:6667 --topic ATLAS_HOOK 2- Send ENTITY_CREATE message : {"version": {"version": "1.0.0"}, "message": {"entities": [{"jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference", "id": {"jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id", "id": "-1467290565135246000", "version": 0, "typeName": "nifi_process_test", "state": "ACTIVE"}, "typeName": "nifi_process_test", "values": {"qualifiedName": "testNifi", "owner": "admin", "description": "Test NIFI", "name": "Test Nifi"}, "traitNames": [], "traits": {} }], "type": "ENTITY_CREATE", "user": "admin"} } I hope it could help you. ... View more

Re: How to create a New Entity in Apache Atlas 0.7...

by Contributor smartdatabundle in Support Questions
‎06-28-2017 09:57 AM
‎06-28-2017 09:57 AM
Hello, You could create a simpe hdfs_path entity : curl -u admin:admin -ik -H "Content-Type: application/json" -X POST -d '{"entity": {"typeName" : "hdfs_path", "attributes" : {"name" : "hdfs_file_test", "qualifiedName" : "testfile@hdpcluster", "path" : "/user/data/test.csv", "clusterName":"hdpcluster"}}}' http://YOUR_HOST:21000/api/atlas/v2/entity ... View more

Atlas : Lineage only with REST API not with Kafka...

by Contributor smartdatabundle in Support Questions
‎06-28-2017 08:40 AM
‎06-28-2017 08:40 AM
Hello, I used the Kafka topic (ATLAS_HOOK) to create a new entity (for hdfs_path). It seems there is no operations to create a lineage between entities and processes (At the current version). Should I use Kafka to create only entities & REST API to create lineage ? Any suggestion ? Thanks. ... View more

Re: Kafka Producer on Apache Atlas via command lin...

by Contributor smartdatabundle in Support Questions
‎06-26-2017 07:40 AM
‎06-26-2017 07:40 AM
Thank you su much @Sharmadha Sainath! Your comment is very useful !!! I tested these commands : $KAFKA_HOME/bin/kafka-console-producer.sh --broker-list MY_HOSTNAME:6667 --topic ATLAS_HOOK Then in Kafka Procuder console: {"version": {"version": "1.0.0"}, "message": {"entities": [{"jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference", "id": {"jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id", "id": "-1467290565135246000", "version": 0, "typeName": "hdfs_path", "state": "ACTIVE"}, "typeName": "hdfs_path", "values": {"qualifiedName": "TestKafka", "owner": "admin", "description": "Test kafka", "path":"/user/data/testkafka.csv", posixPermissions": null, "createTime": "1970-01-01T00:00:00.000Z", "isSymlink": false, "extendedAttributes": null, "numberOfReplicas": 0, "name": "DataKFK"}, "traitNames": [], "traits": {} }], "type": "ENTITY_CREATE", "user": "admin"} } So, Atlas notifies into ATLAS_ENTITIES topic this new entity. I could see it on Atlas UI ! Have a good day ! ... View more

Re: Kafka Producer on Apache Atlas via command lin...

by Contributor smartdatabundle in Support Questions
‎06-23-2017 03:07 PM
1 Kudo
‎06-23-2017 03:07 PM
1 Kudo
Thanks for your comment @Sharmadha Sainath I use HDP SandBox, so I use the default configuration. Mu cluster is not kerberized. First, I tried to use kafka-console-producer.sh to create an entity on ATLAS_HOOK. I had the same error message. Command : lsof -i -P -n | grep kafka Result : java x kafka x IPv6 x 0t0 TCP *:60141 (LISTEN) java x kafka x IPv6 x 0t0 TCP 192.x.x.x:49648->192.x.x.x:2181 (ESTABLISHED) java x kafka x IPv6 x 0t0 TCP 192.x.x.x:6667 (LISTEN) java x kafka x IPv6 x 0t0 TCP 1192.x.x.x:6667->192.x.x.x:32837 (ESTABLISHED) java x kafka x IPv6 x 0t0 TCP 192.x.x.x:47712->192.x.x.x:6667 (CLOSE_WAIT) java x kafka x IPv6 x 0t0 TCP 192.x.x.x:45976->192.x.x.x:6188 (ESTABLISHED) java x kafka x IPv6 x 0t0 TCP 192.x.x.x:6667->192.x.x.x:47717 (ESTABLISHED) Command : $KAFKA_BROKER_HOME/bin/kafka-topics.sh --describe --topic ATLAS_HOOK --zookeeper localhost:2181 Result : Topic:ATLAS_HOOK PartitionCount:1 ReplicationFactor:1 Configs: Topic: ATLAS_HOOK Partition: 0 Leader: 1001 Replicas: 1001 Isr: 1001 ... View more

Kafka Producer on Apache Atlas via command line

by Contributor smartdatabundle in Support Questions
‎06-23-2017 12:37 PM
‎06-23-2017 12:37 PM
Hi, I tried to use this command line : > $KAFKA_BORKER_HOME/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test > $KAFKA_BORKER_HOME/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test When I typed a test message like: test message 1 I had this error message : [2017-06-23 10:17:43,467] ERROR Error when sending message to topic test with key: null, value: 14 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms. Do you have a suggestion ? A this time, I am investigated around Ranger configuration. Thanks for your help. ... View more

Re: Atlas Cross Origin CORS

by Contributor smartdatabundle in Support Questions
‎06-23-2017 12:24 PM
‎06-23-2017 12:24 PM
@Nixon RodriguesThank you so much for your help. It will try it; it should work. May we shoud notify a JIRA ? Or Pull Request on github for this change. It should better if we could put this on atlas-application.properties. Best regards ... View more

Re: Atlas Cross Origin CORS

by Contributor smartdatabundle in Support Questions
‎06-08-2017 07:42 AM
‎06-08-2017 07:42 AM
Hello @Nixon Rodrigues My snippet code : <html> <head> </head> <body> <h1>TEST</h1> <script type="text/javascript"> var myTestRequest = function() { var myUrl = "http://SERVER_IP:21000/api/atlas/types/"; //Instantiate HTTP Request var request = ((window.XMLHttpRequest) ? new XMLHttpRequest() : new ActiveXObject("Microsoft.XMLHTTP")); request.open("GET", myUrl, true); request.setRequestHeader("Content-Type","application/json"); //request.channel.QueryInterface(Components.interfaces.nsIHttpChannel).redirectionLimit = 0; request.setRequestHeader("Authorization", "Basic " + btoa("admin:admin")); //request.withCredentials = true; request.send(null); } </script> <div id="myTestRequest" style="display: block"> <button onclick="myTestRequest()">Login</button> </div> </body> </html> Error (in attached file): XMLHttpRequest cannot load http://SERVER_IP:21000/api/atlas/types/. Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://SERVER_IP:8000' is therefore not allowed access. The response had HTTP status code 400. Thank you for your help. ... View more

Atlas Cross Origin CORS

by Contributor smartdatabundle in Support Questions
‎06-07-2017 03:53 PM
‎06-07-2017 03:53 PM
Hello, I would like to access to Apache Atlas via a XmlHtttRequest object. It doesn't work: Access-Control-Allow-Origin is missing. Thks for your help. ... View more
Labels:

Re: HDFS_PATH in Apache Atlas Search

by Contributor smartdatabundle in Support Questions
‎05-17-2017 09:37 AM
‎05-17-2017 09:37 AM
Hi, The easiest way to find your entity: 1/- On Basic/advanced mode: - select hive_db on Search by Type 2/- Or on advanced mode (DSL), you could filter on Search by Query & just put that: hive_db. To filter by owner, for instance: hive_db where owner="public" ... View more

Re: Anonymous user requests to access on Hive HDFS...

by Contributor smartdatabundle in Support Questions
‎05-15-2017 01:31 PM
‎05-15-2017 01:31 PM
Thank you for your reply. I will try LDAP mode... ... View more

Re: Anonymous user requests to access on Hive HDFS...

by Contributor smartdatabundle in Support Questions
‎05-15-2017 08:53 AM
‎05-15-2017 08:53 AM
Thank you for your reply. Indeed, the value is NONE. ... View more

Anonymous user requests to access on Hive HDFS fil...

by Contributor smartdatabundle in Support Questions
‎05-04-2017 02:47 PM
‎05-04-2017 02:47 PM
Hi all, I installed Hive on HDP 2.5. I configured Ranger to allow access to the HDFS path for only hive user. In Audit Ranger, there is an Anonymous user who wants to use /tmp/hive on HDFS. What is the purpose of this Anonymous user ? Thank you for your advice. ... View more

Re: Data lineage for HDFS files / Hook Atlas for H...

by Contributor smartdatabundle in Support Questions
‎05-02-2017 06:45 AM
‎05-02-2017 06:45 AM
@Ashutosh Mestry Thank you for your reply. Probably it would be interesting to develop a specific bridge for HDFS? ... View more

Data lineage for HDFS files / Hook Atlas for HDFS

by Contributor smartdatabundle in Support Questions
‎04-27-2017 08:30 AM
‎04-27-2017 08:30 AM
Hello, I use HDP 2.6. I would like to create data lineage for Hdfs files with Apache Atlas. The ideal solution uses Apache Atlas to make this data lineage probably via an Hdfs Hook. But this current version seems there is no Hdfs Hook. So I am asking you some advices : - have I create a HDFS entities to manage this lineage ? - Have I create a Hive external table to realize this lineage ? Than you for your help. ... View more

Identity Assertion [regex] in Apache Knox, Ranger

by Contributor smartdatabundle in Support Questions
‎04-27-2017 07:16 AM
‎04-27-2017 07:16 AM
Hello, I defined a list of users in LDAP; uid = user's email address. So, I tried to use this email as username to access to WebHdfs. It doesn't work because of "@" character is illegal argument. Legal argument : ^[A-Za-z_][A-Za-z0-9._-]*[$]?$ Then, I used the Identity Assertion Provider [Regex] to change the user identity: I replace "@" character by "_spe_". It works but there are three issues: - In Ranger it is very difficult to have a clean data audit : in my LDAP I have an email address (bob@test.com) and in Ranger I have (bob_spe_test.com), - My Ranger is also synchronised with LDAP. How Ranger could check in LDAP the user bob_spe_test.com to authorize my real user bob@test.com ? - All my services applied this rule (Identity Assertion Provider). Questions : - I suppose I could define this provider only for Webhdfs. How can I do ? In rewrite.xml file ? It is not documented, I need some advice for this provider. - Is there another way to use an email address as an username ? Thank you for your help. ... View more

Re: Apache Falcon in HDP 3.0

by Contributor smartdatabundle in Support Questions
‎04-26-2017 08:15 AM
‎04-26-2017 08:15 AM
I will recognize that Hortonworks could be a greatest company in Open Source Community mainly in Big Data technologies if Hortonworks could share more details :) In any case, thank you for having taken the time to reply. have a good day, ... View more
Contact Me
Online
Offline
Last Visited
‎08-07-2017 07:18 AM
Public Statistics
Date Registered ‎04-20-2017 02:01 PM
Last Visited ‎08-07-2017 07:18 AM
Total Messages Posted 39
Total Kudos Received 2