About dreamcoding

gkeys · ‎12-02-2016

Not sure specifically what you are looking for, but this shows the capabilities added for each version of atlas http://hortonworks.com/apache/atlas/#section_4

varunprasad_n · ‎07-19-2018

Hi Alex It is Ranger user sync.

wuqi951654775 · ‎09-20-2016

when the job finished,it gived me these messages.But I can't understand it.

mpandit · ‎07-08-2016

Its not required to install Atlas on every node of cluster. You can certianly take a look at http://atlas.incubator.apache.org/HighAvailability.html for high availability options for Apache Atlas.

apathan · ‎07-03-2016

@Ethan Hsieh Atlas client should be available to hive if the HIVE_AUX_JARS_PATH is set to proper location (for local machine installation, it should be set to 'HIVE_AUX_JARS_PATH=<atlas package>/hook/hive'). Can you please check if this is set properly? Please execute below command and find out if it is included in the running hive process just like below. If it is not, please take appropriate action. lsof -p 24932 | grep jar | grep -i atlas java 24932 hive mem REG 254,16 379806 3148643 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/atlas-typesystem-0.7.0.2.5.0.0-831.jar java 24932 hive mem REG 254,16 5137159 3148644 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/kafka_2.10-0.10.0.2.5.0.0-831.jar java 24932 hive mem REG 254,16 3203471 3148637 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/scala-reflect-2.10.4.jar java 24932 hive mem REG 254,16 64723 3148640 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/atlas-client-0.7.0.2.5.0.0-831.jar java 24932 hive mem REG 254,16 745492 3148638 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/kafka-clients-0.10.0.2.5.0.0-831.jar java 24932 hive mem REG 254,16 597252 3148630 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/json4s-core_2.10-3.2.11.jar java 24932 hive mem REG 254,16 57339 3148632 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/hive-bridge-0.7.0.2.5.0.0-831.jar java 24932 hive mem REG 254,16 81388 3148633 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/guice-multibindings-4.0.jar java 24932 hive mem REG 254,16 41144 3148636 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/atlas-common-0.7.0.2.5.0.0-831.jar java 24932 hive mem REG 254,16 84053 3148635 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/json4s-ast_2.10-3.2.11.jar java 24932 hive mem REG 254,16 7126372 3148645 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/scala-library-2.10.4.jar java 24932 hive mem REG 254,16 14445780 3148634 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/scala-compiler-2.10.4.jar java 24932 hive mem REG 254,16 853905 3148642 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/scalap-2.10.4.jar java 24932 hive mem REG 254,16 26484 3148639 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/hdfs-model-0.7.0.2.5.0.0-831.jar java 24932 hive mem REG 254,16 76504 3148641 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/atlas-notification-0.7.0.2.5.0.0-831.jar java 24932 hive mem REG 254,16 69460 3148631 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/json4s-native_2.10-3.2.11.jar java 24932 hive 4r REG 254,16 69460 3148631 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/json4s-native_2.10-3.2.11.jar java 24932 hive 5r REG 254,16 76504 3148641 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/atlas-notification-0.7.0.2.5.0.0-831.jar java 24932 hive 6r REG 254,16 26484 3148639 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/hdfs-model-0.7.0.2.5.0.0-831.jar java 24932 hive 7r REG 254,16 853905 3148642 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/scalap-2.10.4.jar java 24932 hive 8r REG 254,16 14445780 3148634 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/scala-compiler-2.10.4.jar java 24932 hive 9r REG 254,16 7126372 3148645 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/scala-library-2.10.4.jar java 24932 hive 10r REG 254,16 84053 3148635 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/json4s-ast_2.10-3.2.11.jar java 24932 hive 11r REG 254,16 41144 3148636 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/atlas-common-0.7.0.2.5.0.0-831.jar java 24932 hive 12r REG 254,16 81388 3148633 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/guice-multibindings-4.0.jar java 24932 hive 13r REG 254,16 57339 3148632 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/hive-bridge-0.7.0.2.5.0.0-831.jar java 24932 hive 14r REG 254,16 597252 3148630 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/json4s-core_2.10-3.2.11.jar java 24932 hive 15r REG 254,16 745492 3148638 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/kafka-clients-0.10.0.2.5.0.0-831.jar java 24932 hive 16r REG 254,16 64723 3148640 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/atlas-client-0.7.0.2.5.0.0-831.jar java 24932 hive 17r REG 254,16 3203471 3148637 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/scala-reflect-2.10.4.jar java 24932 hive 18r REG 254,16 5137159 3148644 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/kafka_2.10-0.10.0.2.5.0.0-831.jar java 24932 hive 19r REG 254,16 379806 3148643 /grid/0/hdp/2.5.0.0-831/atlas/hook/hive/atlas-typesystem-0.7.0.2.5.0.0-831.jar

dreamcoding · ‎06-29-2016

@Sindhu The table were created in the Hive CLI, and I run import-hive.sh in the dir of $HIVE_HOME/bin. But there is another more important question: https://community.hortonworks.com/questions/41898/using-hive-hook-file-does-not-exist-atlas-client-0.html In the link above, the result showed a error: File does not exist: hdfs://localhost:9000/usr/local/data-governance/apache-atlas-0.7-incubating-SNAPSHOT/hook/hive/atlas-client-0.7-incubating-SNAPSHOT.jar Could you help me to find the file: atlas-client-0.7-incubating-SNAPSHOT.jar ?

vvaks · ‎06-24-2016

@Ethan Hsieh The following labs demonstrate some of the built in hooks that Atlas has with components of HDP. http://hortonworks.com/hadoop-tutorial/tag-based-policies-atlas-ranger/ http://hortonworks.com/hadoop-tutorial/cross-component-lineage-apache-atlas/ Check out this repo for an example of a Nifi Reporting task using the Atlas Java API to capture Nifi provenance in Atlas. https://community.hortonworks.com/content/repo/39432/nifi-atlas-lineage-reporter.html You can use the same approach to integrate any tool that give you the ability to integrate Java code or call out to a REST API to integrate with Atlas. http://atlas.incubator.apache.org/0.6.0-incubating/api/rest.html Check out this thread for a detailed example of how to use a JSON document description of a tag (trait), to register the tag via the Atlas REST API, Add the tag via the UI (this can be done via the REST API as well), and then use the REST API to see the entity with the tag attached as JSON. https://community.hortonworks.com/questions/33501/how-to-create-attribute-sets-and-collections-using.html#answer-40511 Also this one for an example of how to see entities and hive lineage information via the REST API. https://community.hortonworks.com/questions/38380/how-can-we-get-hive-lineage-data-using-rest-api-in.html

dreamcoding · ‎06-30-2016

@Vadim Thank you very much. It really help me to understand the meaning of lineage. And there are two simple question: Q1: If I use Sqoop Bridge, does it mean that I can transmit the metadata from DBMS(eg: MySQL) to Atlas. And it is not necessary to use Sqoop deliver the data from MySQL into Hive, then deliver the data from MySQL into Atlas. In one word, by using Sqoop Bridge, if I can deliver metadata from MySQL to Atlas without Hive ? Q2: When I create another table from the existing hive table in Hive CLI, it will search the JAR files in the HDFS path, but these JAR files are located in the local file system, not in the HDFS. How could I change the path of needed JAR files ? The details of this question is here: https://community.hortonworks.com/questions/41898/using-hive-hook-file-does-not-exist-atlas-client-0.html I hope you can help me. Thank you very much. Thank you.

dreamcoding · ‎06-25-2016

@Ayub Pathan These issues still exist. And I post the descriptions in the last answer, please check/

dreamcoding · ‎06-15-2016

It works. Thank you very much.

Online	Offline
Last Visited	‎12-28-2016 03:42 PM

Member Since	‎04-24-2016 02:18 PM
Last Visited	‎12-28-2016 03:42 PM
Posts	52
Kudos received	11

Cloudera Community

Re: Could anyone send me some documents about the ...

Re: How to set the authority/right of Atlas Web UI...

Re: Sqoop hook doesn't work for atlas?

Re: Must Atlas be installed in every node of a clu...

Re: Using Hive Hook: File does not exist: atlas-cl...

Re: Hive Hook Error: NoClassDefFoundError: com/goo...

Re: How to use the function of data classification...

Re: How to use the audit function of Atlas?

Re: How to import metadata from Hive into Atlas? A...

Re: Why can't I visit Atlas dastboard after run qu...