Member since
09-30-2015
39
Posts
71
Kudos Received
9
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
191 | 11-15-2017 06:41 PM | |
383 | 06-23-2017 07:30 AM | |
269 | 10-07-2016 03:45 AM | |
324 | 10-04-2016 06:32 PM | |
408 | 08-09-2016 11:09 AM |
02-05-2019
07:09 PM
@Giridharan
C
- This is a bug in HDP-3.0 and was fixed in HDP-3.0.1, Ambari-2.7,1 release To workaround the problem, please try the following: Open /etc/hive/conf/hive-env.sh on hive metastore host, change the following line if [ $HADOOP_OPTS != *-Dhive.log.file* ]; then
TO
if [[ "$HADOOP_OPTS" != *-Dhive.log.file* ]]; then and retry the failed task
... View more
05-28-2018
09:00 AM
@S Mahapatra
Few questions:
Can you attach the entire ambari-server.log file? Do you have any mpacks installed through Ambari prior to Upgrade? Have you enabled any experimental features using the 'experimental' flag? Please mention exact HDP-2.6 and Ambari-2.6 versions that you upgraded to
... View more
03-19-2018
08:02 AM
Very cool @Jonas Straub
... View more
03-11-2018
04:32 PM
5 Kudos
Background Starting with Ambari-2.6.1 release, a new repository
has been introduced called HDP-GPL. This article covers FAQs related to the purpose of this repository,
when it takes effect and the impact it can have on your cluster esp. when
dealing with LZO compression libraries. FAQs What is the need for a new GPL repo? LZO compression libraries use a GPL license, which differs from rest of
the packages in Ambari/HDP which are Apache licensed. Hence, the need for a
separate repo which is named ‘HDP-<version_number>-GPL’. The Version
Definition File (VDF) starting HDP-2.6.4.0 release has been updated to include
the new ‘HDP-2.6-GPL’ repo. Refer the sample VDF file for CentOS7 here. Here is a snippet from Ambari web UI for the new 'HDP-2.6-GPL' field: What are the exact contents of HDP-GPL repo? As of HDP-2.6.4.0 release, the repo contains hadooplzo package files as follows: hadooplzo_2_6_4_0_91-0.6.0.2.6.4.0-91.x86_64.rpm hadooplzo_2_6_4_0_91-native-0.6.0.2.6.4.0-91.x86_64.rpm How is Ambari configured to download and install LZO
libraries? For Fresh installs: While running ‘ambari-server setup’, you would be prompted
as follows: Enable Ambari Server to download and install GPL Licensed
LZO packages [y/n] (n)? Depending on your response, a new entry would be created in
ambari.properties file on Ambari server host: ‘gpl.license.accepted=true|false’ For Ambari Upgrades: If you are upgrading to Ambari-2.6.1 (or higher) and have
LZO enabled, you would be prompted to accept GPL license during the ‘ambari
server upgrade’ step [root@myhost yum.repos.d]# ambari-server upgrade
Using python /usr/bin/python
Upgrading ambari-server
INFO: Upgrade Ambari Server
INFO: Updating Ambari Server properties in ambari.properties ...
INFO: Updating Ambari Server properties in ambari-env.sh ...
WARNING: Original file ambari-env.sh kept
INFO: Fixing database objects owner
Ambari Server configured for Embedded Postgres. Confirm you have made a backup of the Ambari Server database [y/n] (y)? y
INFO: Upgrading database schema
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Your cluster is configured to use LZO which is GPL software. You must agree to enable Ambari to continue downloading and installing LZO [y/n] (n)? y
INFO: Return code from schema upgrade command, retcode = 0
INFO: Console output from schema upgrade command:
INFO: {"lzo_enabled":"true"}
INFO: Schema upgrade completed
Adjusting ambari-server permissions and ownership...
Ambari Server 'upgrade' completed successfully.
For both fresh install and upgrades, enabling LZO GPL license is
optional For Express/Rolling Upgrades: Before starting Express/Rolling Upgrade, a pre-check would be performed by Ambari to detect the existence of LZO configs in core-site. In case the GPL license is not accepted and LZO is enabled, you would see the following warning prior to start of Upgrade: You have LZO codec enabled in the core-site config of your cluster. You have to accept GPL license during ambari-server setup to have LZO installed automatically. If any hosts require LZO, it should be installed before starting the upgrade. Consult Ambari documentation for instructions on how to do this. How can I enable LZO GPL license afterwards? ambari-server setup --enable-lzo-under-gpl-license and then restart ambari server service For details on LZO, refer official Ambari documentation here How can I know if GPL license is accepted by Ambari? On Ambari server host, execute below command grep "gpl.license.accepted" /etc/ambari-server/conf/ambari.properties An output of 'gpl.license.accepted=true' implies the license was accepted I do not see the HDP-GPL repo in Manage versions page on
Ambari web UI. Is that expected? Yes, this would be due to non-usage of LZO compression libraries on your
cluster. To confirm, verify if GPL license was denied by Ambari (gpl.license.accepted=false in ambari.properties file) I use a local repository. Does GPL need additional configuration
in such cases? The configuration steps for HDP-GPL local repo are identical
to that of HDP and HDP-UTILS repo. For instance, you would need to first
download the hdp.gpl.repo or the gpl repository
tarball to the selected mirror server. Thereafter, synchronize the repository contents to your mirror server followed by generating the repository metadata: reposync -r HDP-GPL-2.6.4.0
createrepo <web.server.directory>/hdp/<OS>/HDP-GPL-2.6.4.0
For further details on using local repository, refer official
Ambari documentation here What happens when you try to configure core-site with LZO
configs and GPL license is not accepted? Ambari’s stackadvisor has a check to validate such configs. In the current scenario, you would get the following error message: How about some Troubleshooting? ==== Troubleshooting ==== Scenario: Post HDP upgrade, you observe the following error during Pig/MapReduce service check: ERROR 2118: Compression codec com.hadoop.compression.lzo.LzoCodec not found. Cause: Your have LZO configured, but not accepted the GPL
license during Ambari upgrade Resolution: Enable GPL license using "ambari-server setup --enable-lzo-under-gpl-license" command ==== Bug Fixes ==== AMBARI-22906 - Fixed in Ambari-2.6.1.5 release AMBARI-23055 - Fixed in the upcoming Ambari-2.6.2.0 release
... View more
- Find more articles tagged with:
- ambari-server
- Cloud & Operations
- FAQ
- gpl
- upgrade
Labels:
03-10-2018
09:07 PM
@Muhammad Touseef - Assuming you are using Ambari-2.6.1 and HDP-2.6.4, is GPL license accepted during Ambari server setup. To verify on Ambari server host, execute below command > grep gpl /etc/ambari-server/conf/ambari.properties If you see an output as 'gpl.license.accepted=false' ; GPL is not accepted. To enable GPL, run: ambari-server setup --enable-lzo-under-gpl-license Thereafter refer docs to enable LZO: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.1.5/bk_ambari-administration/content/ch_amb_ref_configuring_lzo_compression.html Hope this helps! - Vivek
... View more
03-10-2018
09:00 PM
@Muhammad Touseef - Assuming you are using HDP-2.6.4.0 and Ambari-2.6.1 version, is GPL license enabled in your Ambari server? To verify, execute below command on Ambari server host: > grep gpl /etc/ambari-server/conf/ambari.properties If the output says: gpl.license.accepted=true Then GPL is enabled, else it isn't. If GPL is not enabled, run below command from Ambari server host: ambari-server setup --enable-lzo-under-gpl-license Once GPL is enabled, refer Ambari docs related to LZO: https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.1.5/bk_ambari-administration/content/ch_amb_ref_configuring_lzo_compression.html Hope this helps! - Vivek
... View more
03-09-2018
05:41 PM
@Jonas Straub - Nice article! Can you please update the commands with the following additional service checks? RANGER_KMS_SERVICE_CHECK, AMBARI_INFRA_SERVICE_CHECK, KERBEROS_SERVICE_CHECK, SLIDER_SERVICE_CHECK
... View more
11-15-2017
06:41 PM
2 Kudos
@William Lu - yes, you would need to delete Atlas service and install it back after upgrade to HDP-2.6.0 Atlas was in Tech Preview mode in earlier versions of HDP like v2.3 and v2.4, so once you install new Atlas post upgrade - it will be a full blown GA'ed version that you will get. More about Atlas here Hope this helps! ------ Vivek
... View more
09-04-2017
07:04 PM
4 Kudos
@ARUN - You may be using an older version of Ambari HDF and HDP work together starting Ambari-2.5.1.0 with HDP-2.6.1.0 Please see below links for details: https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.0/bk_installing-hdf-on-hdp/content/ch_add-hdf-to-hdp.html https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.0/bk_installing-hdf-on-hdp/content/upgrade_ambari.html In this case, you will have to upgrade Ambari first to 2.5.1.0 and then HDP to 2.6.1.0. Thereafter try out installing the HDF services using the recommended steps in the doc Hope this helps! - Vivek
... View more
06-26-2017
06:32 PM
@Aaron Norton - Please accept the answer if you think it was relevant to your question
... View more
06-23-2017
07:30 AM
1 Kudo
@Aaron Norton - Looks like the knox.py script on your cluster is still referring to an older parameter named 'recursive' There was a fix done as part of AMBARI-14396 where the parameter was modified from 'recursive' to 'create_parents' Can you try the following? On your knox gateway host open this file: /var/lib/ambari-agent/cache/common-services/KNOX/0.5.0.2.2/package/scripts/knox.py Change to the following under line having text: os.path.join(params.knox_conf_dir, "topologies")], owner = params.knox_user,
group = params.knox_group,
create_parents = True,
cd_access = "a",
mode = 0755,
recursive_ownership = True, Ensure that the indentation is retained See sample entry here Thanks, Vivek
... View more
05-08-2017
03:39 PM
10 Kudos
Ambari version 2.5 provides a new Upgrade History page. This
is useful in many ways such as: Determine the history of upgrades that your
cluster has undergone (and any issues). Collect any specific logs from within the web UI
without the need to login to the respective host. Determine time taken for upgrade, which gives an
indication of the time required for the next upgrade. In short, Upgrade planning and troubleshooting are two
aspects addressed through this page Here is how the ‘Upgrade History’ page looks: Besides the above UI, you could also retrieve
the entire Upgrade output through a couple of Ambari API calls http://<ambari_server>:8080/api/v1/clusters/<cluster_name>/upgrades The above call provides the list of Upgrade ids.
The highest id in the list is usually the most recent Upgrade on the cluster.
This upgrade_id serves as an input to the second call as follows: http://<ambari_server>:8080/api/v1/clusters/<cluster_name>/upgrades/<upgrade_id>?fields=upgrade_groups/upgrade_items/tasks/Tasks/status,upgrade_groups/upgrade_items/tasks/Tasks/command_detail,upgrade_groups/upgrade_items/tasks/Tasks/stderr,upgrade_groups/upgrade_items/tasks/Tasks/host_name The second call provides the entire Upgrade
output grouped by Upgrade Groups, Upgrade Items and finally the Upgrade Tasks.
You could extend the call by including additional fields as they appear in an
Upgrade task output as below: Additionally you could redirect the output of above call to a file and send it over for troubleshooting any issues.
... View more
- Find more articles tagged with:
- Ambari
- Cloud & Operations
- express
- express-upg
- How-ToTutorial
- issue-resolution
- rolling
- rolling-upg
- upgrade
Labels:
03-08-2017
04:33 PM
3 Kudos
@Artem Ervits According to the upgrade instructions from 2.4.2 http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-upgrade/content/upgrade_ambari.html You upgrade server, then agent and then upgrade the DB schema - 'ambari-server upgrade' The order is important and should be the same for 2.5.0 as well
... View more
02-22-2017
06:31 PM
1 Kudo
@kotesh banoth May also want to check below document for rectifying the pre-check errors: https://community.hortonworks.com/storage/attachments/7372-ru-eu-prechecks-details.pdf Source page: https://community.hortonworks.com/articles/54944/stack-upgrade-pre-checks-purpose-and-remediation.html
... View more
12-09-2016
06:50 AM
3 Kudos
@gsharma - I generally refer https://community.hortonworks.com/articles/2473/rolling-upgrade-express-upgrade-in-ambari.html
... View more
11-30-2016
12:26 PM
2 Kudos
@Mark - Were there any failures during the upgrade? If yes, how did you fix them? Can you paste a screenshot of your Upgrade Wizard? As for finalize, it would be permitted when Ambari knows that all components on all hosts have been successfully upgraded. You can hit 'Finalize Later' (equivalent to Pause Upgrade) and try the following:
Run 'ambari-server restart' on Ambari server host Restart all services from Ambari UI Restart Ambari agents on both hosts And then Resume upgrade and try to Finalize. This may/may not solve the issue depending on what the problem was during your upgrade The other option would be to downgrade your cluster (since I see majority of the components do not show 2.4.3 version) and start upgrade afresh -Vivek
... View more
11-13-2016
06:33 PM
2 Kudos
@zkfs Assuming you have upgraded to Ambari 2.2.0.0 you could perform Express Upgrade of HDP from 2.1 to 2.3 --> this is a supported path See http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.0.0/bk_upgrading_Ambari/content/_upgrading_HDP_perform_express_upgrade.html for more info You could use a local repository for the upgrade. Since upgrade is for HDP, you would need to setup HDP local repo (and not Ambari) for the upgrade. This link has the details: http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.0.0/bk_Installing_HDP_AMB/content/_using_a_local_repository.html
... View more
10-07-2016
03:45 AM
2 Kudos
@Eric Krenz
Try this one to start Demo LDAP curl -i -uadmin:admin -H 'X-Requested-By: ambari' -X POST -d '{"RequestInfo":{"context":"Start Demo LDAP","command":"STARTDEMOLDAP"},"Requests/resource_filters":[{"service_name":"KNOX","component_name":"KNOX_GATEWAY","hosts":"$knox_gateway_hostname"}]}' http://$ambari_server:8080/api/v1/clusters/$cluster_name/requests To stop Demo LDAP curl -i -uadmin:admin -H 'X-Requested-By: ambari' -X POST -d '{"RequestInfo":{"context":"Stop Demo LDAP","command":"STOPDEMOLDAP"},"Requests/resource_filters":[{"service_name":"KNOX","component_name":"KNOX_GATEWAY","hosts":"$knox_gateway_hostname"}]}' http://$ambari_server:8080/api/v1/clusters/$cluster_name/requests
... View more
10-04-2016
06:32 PM
1 Kudo
@Saurabh Looks like the HDP-UTILS repo path is incorrect. I see an additonal "/hdp/" in your path The UTILS repo for CentOS6 is at: http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6 The UTILS tar file is at: http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/centos6/HDP-UTILS-1.1.0.20-centos6.tar.gz Check this link for more
... View more
09-12-2016
05:23 AM
Great article @Alejandro Fernandez Have added more details about the pre-checks here - https://community.hortonworks.com/articles/54944/stack-upgrade-pre-checks-purpose-and-remediation.html
... View more
09-09-2016
10:38 AM
4 Kudos
@Arun A K - You may want to check the output of below REST call curl -u admin:admin -i -H 'X-Requested-By: ambari' -X GET http://$server:8080/api/v1/clusters/$clustername/configurations/service_config_versions This would list the service config versions of all services. At any time you make the above call, it would provide the complete history of service config changes. Some fields that could be of interest are mentioned in below post https://community.hortonworks.com/questions/45211/how-to-extract-configuration-changes-from-ambari.html#answer-45253 Hope this helps!
... View more
09-05-2016
11:25 PM
5 Kudos
Cluster upgrade is a well-planned maintenance activity.
Before starting the upgrade, it is important to ensure that your cluster is
ready for upgrade. Ambari has built-in pre-checks that are executed before the
actual upgrade to check the overall health of your cluster and validate its readiness for upgrade. Let us look at the pre-checks, their purpose and how to
remediate the issues, so that your cluster upgrade process is smooth. A bit of background first: Background Ambari supports two kinds of stack (cluster) upgrades: Rolling
Upgrade (RU) and Express Upgrade (EU). Pre-checks apply to both upgrade
methods, although RU has a larger set of pre-checks since the upgrade does not
involve any downtime First up, the types of pre-checks:
Cluster level checks: Example – newer version
packages should be installed on all hosts in the cluster Service level checks: Example – all service
components should be in installed state (no failures) Host level checks: All hosts must be
communicating with Ambari (no network issues, no heartbeat failures from Ambari
agent to server) The output of a pre-check would result in three possible
values of pre-check status:
ERROR WARNING PASS ERRORS that have to be fixed before the upgrade
can be started. WARNINGS are recommended to be fixed, but may be ignored, if
you know what you are doing When you hit ‘Upgrade’ button in Ambari User Interface (UI) – the pre-checks
are kicked off in the background and within a matter of seconds you would be
notified if there are some issues that need to be fixed before upgrade (RU or
EU) can be started. Here is a snippet from Pre-Checks Window in Ambari UI: Behind the scenes, below API call is triggered: curl -u $user:$password -i -H 'X-Requested-By: ambari' -X GET http://$server:8080/api/v1/clusters/$name/rolling_upgrades_check?fields=*&UpgradeChecks/repository_version=$version&UpgradeChecks/upgrade_type=ROLLING
- $user: Ambari admin
user - $password: Password
of the Ambari admin user account - $server: Ambari
server hostname or IP address - $name: Cluster name - $version: The actual
stack version you wish to upgrade to; example: 2.5.0.0-1245 (this must be
registered beforehand and package installation should be completed) - upgrade_type: ROLLING
for Rolling Upgrade, NON_ROLLING for Express Upgrade
============================================================================ Details about the
pre-checks and remediation
Refer ru-eu-prechecks-details.pdf Table above is updated as of Ambari 2.4 release
... View more
- Find more articles tagged with:
- Ambari
- Cloud & Operations
- How-ToTutorial
- rolling-upg
- stack-upgrade
- upgrade
Labels:
09-01-2016
06:24 AM
@Ramakrishna Pratapa - Can you attach ambari-server-check-database.log file present in /var/log/ambari-server/ directory? Also please try running: ambari-server start --skip-database-check and let us know how it goes From the first look, the issue appears due to an inconsistency in a service configuration. Additionally, please paste the output of: ambari-server --version -Vivek
... View more
08-26-2016
07:02 PM
2 Kudos
@Saifullah Sajjad - While we await your error log, you just have to place ambari.repo on ambari server node. hdp.repo is not required on any of the nodes as Ambari server takes care of pushing the repos on all nodes that you designate as part of the cluster.
... View more
08-10-2016
06:16 PM
2 Kudos
@Zach Kirsch Check this article from @Kuldeep Kulkarni which has a bunch of scripts that would suit your requirement https://community.hortonworks.com/articles/34747/ambari-admin-utility.html ./ambari-admin.sh restart refresh is the one The source code of the above script as mentioned in the article is available at: https://github.com/crazyadmins/useful-scripts/blob/master/ambari/ambari-admin.sh -Vivek
... View more
08-09-2016
12:59 PM
1 Kudo
@Narasimha Gunturu - In such cases, it is good to write a script to upgrade the agents across all nodes For example: key="/tmp/ambari-keypair"
for i in {1..5} ; do ssh -i $key -o StrictHostKeyChecking=no root@host-$i.mydomain ambari-agent stop ; done
repourl="http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.2.2.0/ambari.repo"
for i in {1..5} ; do ssh -i $key root@host-$i.mydomain wget -nv $repourl -O /etc/yum.repos.d/ambari.repo; done
for i in {1..5} ; do ssh -i $key root@host-$i.mydomain yum upgrade -y ambari-agent ; done
for i in {1..5} ; do ssh -i $key root@host-$i.mydomain ambari-agent start ; done
Here host-1.mydomain to host-5.mydomain are the five agent nodes and 'key' variable holds the ssh key to connect to those nodes from a central server (could be the Ambari server host)
... View more
08-09-2016
11:09 AM
2 Kudos
@Narasimha Gunturu You would need to upgrade the Ambari agent in other nodes as well. Check below link for details (point #7): http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_upgrading_Ambari/content/_upgrade_ambari.html Once agents are upgraded and started, they would sync up with Ambari server. All agents should be at same version as Ambari server -Vivek
... View more
08-03-2016
02:34 PM
@Matthias Rueling Can you paste the screenshot of Javascript console window when you click on "Perform Upgrade"? This can be enabled in Chrome browser - View - Developer - Developer Tools - Network tab. It would show the API call (generally the upgrade pre-check call, I think) that is getting invoked and errors if any, depending on its return code The other entries to check would be in /var/log/ambari-server/ambari-server.log on Ambari host. Based on the above two, we can understand the issue and then decide next action -Vivek
... View more
08-03-2016
10:05 AM
1 Kudo
@Jose A. Gomez One thing I see missing in your alert json is uri In the link you mentioned, there is an example of uri as: "uri" : "{{core-site/ha.zookeeper.quorum}}" Please see if setting a value for uri corresponding to Flink resolves the issue -Vivek
... View more