Member since
08-24-2018
91
Posts
5
Kudos Received
3
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 6209 | 09-16-2021 06:57 AM | |
| 2938 | 01-08-2021 04:40 AM | |
| 1528 | 03-25-2020 10:07 PM |
09-24-2021
05:46 AM
Glad that the issue has been resolved @dansteu. Can you please accept my answer as a solution: https://community.cloudera.com/t5/Support-Questions/Data-Lineage-not-shown-after-importing-Hive-entities/m-p/324554/highlight/true#M229402 This will help the community find answers in future. Thanks!
... View more
09-23-2021
05:17 AM
@dansteuCan you share the output of these commands? Please run them from the Kafka broker host and request the Kafka principal prior: /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --list --zookeeper `hostname -f`:2181 And the second command: https://gitlab.com/saurabhlambe/Atlas-stuff/-/blob/master/Atlas%20API.md#c-describe-atlas-consumer-group-to-check-current-consumer-lag
... View more
09-17-2021
08:09 AM
@dansteuyou are correct. Only if Atlas fetches Hive metadata automatically (with no manual intervention), only then the lineage is created. Please refer to this Apache Atlas document to set up the Atlas Hive hook: https://atlas.apache.org/1.2.0/Hook-Hive.html BTW, when you install Atlas in an HDP cluster, these properties are automatically set and no intervention is needed. So you might just need to confirm if those properties are set. Regards, Saurabh
... View more
09-16-2021
06:57 AM
Hi @dansteu, First of all, executing the import-hive.sh command will only import Hive entities (DBs, tables, columns) into Atlas. It won't create the inter-table lineage. The only way to see lineage is for Atlas to automatically fetch those entities from Hive using the Atlas hook. In order to see lineage, you need the following components in a healthy state: Hive, Hbase, Kafka, Solr, Atlas. The article you referred to is correct and note that it doesn't ask you to run import-hive.sh at all! Hope this helps. Regards, Saurabh
... View more
04-19-2021
06:59 AM
Hi Megh, Looking at the error, it seems that a NN failover happened at the remote cluster when the distcp didn't yet finish. Also, did you mention namespaces instead of hostnames in the distcp command? I would recommend you review this document on performing distcp between clusters that have Namenode HA enabled. Thanks, Saurabh
... View more
01-08-2021
04:40 AM
Hi @doronve, I believe you want to set up a local repository that does not need an internet connection? Please review this document for more information: https://docs.cloudera.com/cdp-private-cloud-base/7.1.4/installation/topics/cdpdc-local-package-parcel-repositories.html
... View more
03-25-2020
10:07 PM
You do not need Spark running to see Hive table lineage in Atlas. However, if you're using the Spark hook, you must have Spark turned on.
... View more
01-07-2019
02:26 PM
2 Kudos
Problem statement: How to upgrade Apache Ambari 2.6.2 to Apache Ambari 2.7.3
Abstract: Apache Ambari 2.7.3 is the latest among Ambari 2.7.x releases. Ambari 2.7.0, which was the first release in the 2.7.x series introduced significant improvements from its predecessor - Ambari 2.6.2. This tutorial will help users upgrade from Ambari 2.6.2.2 to Ambari 2.7.3
Procedure:
I. Prerequisites
1. Backup Ambari configuration file
# mkdir /root/backups
# cp /etc/ambari-server/conf/ambari.properties /root/backups
2. Turn off Service Auto Restart From Ambari UI: Admin > Service Auto Start. Set Auto Start Services to Disabled. Click Save.
3. Run Service Checks on all Ambari services On each of the Ambari services installed on your cluster, Run Service Checks as shown below: From Ambari UI: <Service_Name> > Service Actions > Run Service Check For example: HDFS > Service Actions > Run Service Check 4. Start and Stop all of the Ambari services from Ambari UI. II. Stop Services 1. If
SmartSense is deployed, stop it and turn on Maintenance Mode
–From
Ambari Web,
browse to Services > SmartSense and
select Stop from
the Service Actions
menu. Then, select Turn
on Maintenance Mode from
the Service Actions
menu. 2. If
Ambari Metrics is deployed, stop
it and turn on Maintenance Mode
–From
Ambari Web,
browse to Services > Log Search
and select Stop from
the Service Actions
menu. Then, select Turn
on Maintenance Mode from
the Service Actions
menu. 3. If
you are running Log Search in your cluster, stop the service.
–From
Ambari Web,
browse to Services > Log Search
and select Stop from
the Service Actions
menu. Then, select Turn
on Maintenance Mode from
the Service Actions
menu. 4. Stop Ambari server # ambari-server stop 5. Stop Ambari agents # ambari-agent stop 6. Backup Ambari database # mysqldump -u ambari -p ambari > /root/backups/ambari-before-upgrade.sql
III. Download Ambari 2.7.3 repository
1. Replace the old Ambari repository with the latest on on all hosts in the cluster # wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari.repo -O /etc/yum.repos.d/ambari.repo 2. Upgrade Ambari server # yum clean all
# yum upgrade ambari-server Important:If you have deployed HDF components in your HDP setup, you need to upgrade the HDF Management Pack before upgrading the Database Schema in step IV: Upgrade the HDF Management Pack 3. Upgrade Ambari agents # yum clean all
# yum upgrade ambari-agent
IV. Upgrade Database Schema 1. On the Ambari server host, upgrade the Ambari database schema # ambari-server upgrade 2. Start Ambari server # ambari-server start 3. Start Ambari agents # ambari-agent start
V. Verify Ambari version From the Ambari UI: Go to Admin > About
... View more
Labels: