Member since
09-14-2021
19
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
509 | 10-05-2021 12:49 AM |
11-29-2021
05:07 AM
Hi @dansteu The kerberos service name property has to be the service name specified for the kafka service which is usually "kafka".
... View more
10-27-2021
08:43 AM
Hi @slambe , still related to this thread, I would like to know how it's possible to see data lineage of an existing Hive DB. Better to explain me; if I execute the data lineage executing the scripts of the article https://community.cloudera.com/t5/Community-Articles/Using-Apache-Atlas-to-view-Data-Lineage/ta-p/246305 it works perfectly now. But imagine to suggest this solution to an existing prod environment with tons of tables, relations - the scripts of creation are hidden to me but I would guess Atlas can read table fields and dependencies; how could the data lineage be available in Atlas in this case? the logic between tables is hidden to Atlas cause the creation of structures has been defined during Hive DB creation, years ago, and it's not a live process. Thanks, Best Regards, Daniele.
... View more
10-05-2021
01:53 PM
@dansteu
There are multiple changes needed in the properties.
The Atlas Configuration Directory shouldn't be on the /tmp directory. The ideal location would be a directory in the default class path of NiFi.
Create Atlas Configuration File should be set to true
If Kerberos is being used for authentication, why is the URL http and not https?
For the NIFi URL for Atlas, it would be better to use http://${hostname(true)}:8080/nifi this will provide a URL that will map back to the NiFi component that generated the Atlas lineage
The Atlas Default Metadata Namespace should be populated with a value that is representative of the flow. For example, For example, hdf352 or testflowversion1, or something similar.
... View more
10-05-2021
12:49 AM
Thanks @MattWho ! It works. The feature "ReportLineageToAtlas" is now enabled in NiFi. Thanks a lot.
... View more
10-04-2021
05:24 AM
Hello Team, @ahadjidj Running this suggested command mvn clean install -Pinclude-atlas -DskipTests referrinng to the pom xml file here located: /work/nar/framework/nifi-framework-nar-1.14.0.nar-unpacked/META-INF/maven/org.apache.nifi/nifi-framework-nar/pom.xml I get this error message : [WARNING] The requested profile "include-atlas" could not be activated because it does not exist. [ERROR] Failed to execute goal org.apache.nifi:nifi-nar-maven-plugin:1.3.1:nar (default-nar) on project nifi-evtx-nar: The plugin org.apache.nifi:nifi-nar-maven-plugin:1.3.1 requires Maven vers ion 3.1.0 -> [Help 1] I got this message: The requested profile "include-atlas" could not be activated because it does not exist. Could you please provide any hint? As we need to introduce the lineage into Atlas with info coming from Nifi, this area in currently on priority as we are stucked on this. Thanks a lot! Daniele.
... View more
09-16-2021
03:42 AM
The answer would be no. @dansteu I once came into a support case. A customer wanted to use NiFi to integrate with an external Ranger, which means that NiFi and Ranger are not under the same Ambari management. Ranger's SME told us that this is impossible. At least this is not within the scope of Cloudera's technical support. The logic should be the same for Atlas. However, if you know enough about Ranger and Ambari, it should still be possible technically in theory.
... View more
09-16-2021
12:20 AM
Hi, I created yesterday also this new thread https://community.cloudera.com/t5/Support-Questions/Data-Lineage-not-shown-after-importing-Hive-entities/td-p/324441 Thanks in advance for support. Best Regards, Daniele.
... View more
- Tags:
- apache atlas
09-15-2021
03:40 AM
Hi, I am getting exactly same issue. As a consequence, the data lineage is not shown in Atlas after import-hive.sh script is executed. Did you resolve this topic? Could you please give me any hint? Thanks a lot!
... View more
09-14-2021
09:09 AM
Hi, I know this question is an old one, but I am exactly in this situation. I have Cloudera and Hive that has been filled through Hue. I created a simple data structure with a couple of tables, then joined them in order to create a third table. I then run the script to import these data into Atlas. I figure out now in this forum that import-hive script is not supporting data lineage. Indeed after script execution, data lineage for tables is empty. How can I then migrate data from Hive into Atlas, including the data lineage? Thanks a lot for your answer.
... View more