Member since
05-30-2018
1322
Posts
715
Kudos Received
148
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 4067 | 08-20-2018 08:26 PM | |
| 1962 | 08-15-2018 01:59 PM | |
| 2390 | 08-13-2018 02:20 PM | |
| 4138 | 07-23-2018 04:37 PM | |
| 5045 | 07-19-2018 12:52 PM |
11-17-2016
05:07 AM
@Markus Maus I do not recommend going directly against hbase. Use Kafka or the api. can you describe why you are not using kafka or the api?
... View more
11-16-2016
05:11 AM
1 Kudo
@Markus Maus Atlas integration is highly flexible and you can use rest or kafka. For HDP partners, those tools integrate with Atlas using hooks. For others simply consume and publish all metadata using kafka. I would go directly against hbase. Use kafka topic to publish and consume metadata. Or you can use RestApi. Both options are documented on apache atlas http://atlas.incubator.apache.org/AtlasTechnicalUserGuide.pdf There are two types of messages for which Kafka is used. Each type is written to a
specific topic in Kafka. ● Publishing entity changes to Atlas: These messages are passed from the
metadata sources where the metadata is originally created / updated or
deleted to Atlas. These messages are written to a topic called ATLAS_HOOK.
Typically, these metadata sources are other components in the Hadoop
ecosystem. As of Atlas 0.7-incubating, there are integrations with Hive, Sqoop,
Falcon and Storm with Atlas. ● Consuming entity changes from Atlas: These messages are passed from Atlas
to external consumers who might be interested in changes to metadata. An
example of such a source in the current Hadoop ecosystem is Apache Ranger.
By capturing metadata change events in real time, Ranger provides policy
driven security management of Hadoop data assets. These messages are
written to a topic called ATLAS_ENTITIES.
... View more
11-16-2016
02:51 AM
can you confirm java is installed here /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/bin/java
... View more
11-15-2016
06:29 AM
what is your primary key and is the primary key in the where clause? did you create local or global index?
... View more
11-15-2016
06:25 AM
1 Kudo
here is example to drop partition ALTER TABLE mytable DROP IF EXISTS PARTITION(year = 2012, month = 12, day = 18);
... View more
11-15-2016
06:19 AM
is your cluster kerberized? do you have ranger enabled? is ranger plugin enabled for kafka? if yes to those questions, does nifi user have access to topic you are trying to consume from? Also confirming you are using consumekafka and not the consumekafka_0_10 version?
... View more
11-14-2016
09:12 PM
1 Kudo
Is dynamic query rewrite avialable for hive as it is In HDP 2.5 dynamic query rewrite via Ranger for SparkSQL?
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Ranger
-
Apache Spark
11-14-2016
04:00 PM
2 Kudos
I found how to do it
Identity Propagation with JDBC (Hive)
1.Enable
Authentication via Shiro
Config
–zeppelin.jdbc.auth.type
= KERBEROS
–zeppelin.jdbc.principal
=
principal value
–zeppelin.jdbc.keytab.location
=
keytab
location
•JDBC
Interpreter will add end-user as
jdbc:hive2://HiveHost:10000/default;principal=hive/_HOST@MY-REALM.COM;hive.server2.proxy.user=<$USER_NAME>
... View more
11-14-2016
03:56 PM
1 Kudo
Is there any capabilities to hive zeppelin notebook run hive query based on user id on zeppelin session? right know the jdbc interrupter needs username and password for hive. That means all users will run as user defined in interpreter settings. I can't have this due to security issues, ie having all users access hive tables via single user id. I have ranger enabled to provide security based on user id. Livy is for spark so that is unusable for hive. any insights?
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Ranger
-
Apache Zeppelin
11-11-2016
08:58 PM
Ranger has audit logs for when users run hive queries. I need the ability to determine which yarn job id is that audit assoicatd with. Any insights? I created this post: https://community.hortonworks.com/users/1486/smanjee.html#1486-activity-block but no response on the application ID/Job id side.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Ranger