Member since
08-24-2018
91
Posts
5
Kudos Received
3
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 6210 | 09-16-2021 06:57 AM | |
| 2941 | 01-08-2021 04:40 AM | |
| 1529 | 03-25-2020 10:07 PM |
10-27-2021
08:43 AM
Hi @slambe , still related to this thread, I would like to know how it's possible to see data lineage of an existing Hive DB. Better to explain me; if I execute the data lineage executing the scripts of the article https://community.cloudera.com/t5/Community-Articles/Using-Apache-Atlas-to-view-Data-Lineage/ta-p/246305 it works perfectly now. But imagine to suggest this solution to an existing prod environment with tons of tables, relations - the scripts of creation are hidden to me but I would guess Atlas can read table fields and dependencies; how could the data lineage be available in Atlas in this case? the logic between tables is hidden to Atlas cause the creation of structures has been defined during Hive DB creation, years ago, and it's not a live process. Thanks, Best Regards, Daniele.
... View more
04-22-2021
03:22 AM
Understood. In our case, the culprit turned out to be 50075 port in firewall. I've added as an answer. In any case, thanks for your support. Thanks, Megh
... View more
01-08-2021
07:46 AM
CM uses BitTorrent to distrubte parcels so it's already very efficient. My recommendation would be to let CM do the work and not bother with trying to save yourself a few minutes.
... View more
03-25-2020
10:07 PM
You do not need Spark running to see Hive table lineage in Atlas. However, if you're using the Spark hook, you must have Spark turned on.
... View more
01-07-2019
02:26 PM
2 Kudos
Problem statement: How to upgrade Apache Ambari 2.6.2 to Apache Ambari 2.7.3
Abstract: Apache Ambari 2.7.3 is the latest among Ambari 2.7.x releases. Ambari 2.7.0, which was the first release in the 2.7.x series introduced significant improvements from its predecessor - Ambari 2.6.2. This tutorial will help users upgrade from Ambari 2.6.2.2 to Ambari 2.7.3
Procedure:
I. Prerequisites
1. Backup Ambari configuration file
# mkdir /root/backups
# cp /etc/ambari-server/conf/ambari.properties /root/backups
2. Turn off Service Auto Restart From Ambari UI: Admin > Service Auto Start. Set Auto Start Services to Disabled. Click Save.
3. Run Service Checks on all Ambari services On each of the Ambari services installed on your cluster, Run Service Checks as shown below: From Ambari UI: <Service_Name> > Service Actions > Run Service Check For example: HDFS > Service Actions > Run Service Check 4. Start and Stop all of the Ambari services from Ambari UI. II. Stop Services 1. If
SmartSense is deployed, stop it and turn on Maintenance Mode
–From
Ambari Web,
browse to Services > SmartSense and
select Stop from
the Service Actions
menu. Then, select Turn
on Maintenance Mode from
the Service Actions
menu. 2. If
Ambari Metrics is deployed, stop
it and turn on Maintenance Mode
–From
Ambari Web,
browse to Services > Log Search
and select Stop from
the Service Actions
menu. Then, select Turn
on Maintenance Mode from
the Service Actions
menu. 3. If
you are running Log Search in your cluster, stop the service.
–From
Ambari Web,
browse to Services > Log Search
and select Stop from
the Service Actions
menu. Then, select Turn
on Maintenance Mode from
the Service Actions
menu. 4. Stop Ambari server # ambari-server stop 5. Stop Ambari agents # ambari-agent stop 6. Backup Ambari database # mysqldump -u ambari -p ambari > /root/backups/ambari-before-upgrade.sql
III. Download Ambari 2.7.3 repository
1. Replace the old Ambari repository with the latest on on all hosts in the cluster # wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari.repo -O /etc/yum.repos.d/ambari.repo 2. Upgrade Ambari server # yum clean all
# yum upgrade ambari-server Important:If you have deployed HDF components in your HDP setup, you need to upgrade the HDF Management Pack before upgrading the Database Schema in step IV: Upgrade the HDF Management Pack 3. Upgrade Ambari agents # yum clean all
# yum upgrade ambari-agent
IV. Upgrade Database Schema 1. On the Ambari server host, upgrade the Ambari database schema # ambari-server upgrade 2. Start Ambari server # ambari-server start 3. Start Ambari agents # ambari-agent start
V. Verify Ambari version From the Ambari UI: Go to Admin > About
... View more
Labels: