Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

RPM missing files during install

Highlighted

RPM missing files during install

Hello,

I am having issues during HDP install. Services have trouble coming online because some RPM's have files missing when after they are installed. For instance. Yarn NodeManager would fail to come online cause of a missing spark-yarn-shuffle.jar. The RPM that is supposed to contain the jar is installed according to yum. Removing that RPM using with --no-deps option and reinstalling it fixes this problem. The RPM uninstall shows what files were missing which includes the jar in question. The same thing happens when bringing up the Kafka brokers (KafkametricsSink jar) There is no guarantee that the same services fail each time but the same culprits tend to fail more often than not.

This may or may not be an Ambari or HDP related issue but I am asking if anyone encountered similar issues and how they ensured that the HDP services install was copacetic? Any ideas on what I could check are also welcome.

The servers do not have internet access and a local repo was setup for us.

10 REPLIES 10
Highlighted

Re: RPM missing files during install

Mentor

@KMan

I would advice you start a fresh install and make sure you have the correct version of hdp.repo and ambari.repo

yum clean all
yum repolist 

Once the above is correct you can proceed with the Ambari-server and Ambari-agent install.

Highlighted

Re: RPM missing files during install

Thanks for the quick response.

I shall try it out but it doesn't answer my questions. Why would only part of the files that comprise an RPM be copied to their appropriate locations? Are these installation errors logged somewhere ?

Highlighted

Re: RPM missing files during install

Mentor

@KMan

If you are on Linux run

# yum

Without and arguments it will give you options like"Check,makecache,deplist " below to mention but a few that could help you

check          Check for problems in the rpmdb
check-update   Check for available package updates
clean          Remove cached data
deplist        List a package's dependencies
list           List a package or groups of packages
load-transaction load a saved transaction from filename
makecache      Generate the metadata cache
provides       Find what package provides the given value
reinstall      reinstall a package
repolist       Display the configured software repositories
resolvedep     Determine which package provides the given dependency
search         Search package details for the given string
Highlighted

Re: RPM missing files during install

This is a sample output from running yum verify-all on one of my cluster data nodes.

Any ideas about why I am hitting this issue repeatedly? The RPM was installed with files missing. A reinstall fixes the issues. So why does it fail during its first install?

ambari-metrics-hadoop-sink.x86_64 : Ambari Metrics Assembly
    File: /usr/lib/ambari-metrics-hadoop-sink
        Problem:  file is missing
    File: /usr/lib/ambari-metrics-hadoop-sink/ambari-metrics-hadoop-sink-with-common-2.2.2.0.460.jar
        Problem:  file is missing
    File: /usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink-with-common-2.2.2.0.460.jar
        Problem:  file is missing
    File: /usr/lib/ambari-metrics-kafka-sink/lib
        Problem:  file is missing
    File: /usr/lib/ambari-metrics-kafka-sink/lib/commons-lang-2.6.jar
        Problem:  file is missing
    File: /usr/lib/ambari-metrics-kafka-sink/lib/commons-logging-1.1.1.jar
        Problem:  file is missing
    File: /usr/lib/ambari-metrics-kafka-sink/lib/jackson-core-asl-1.9.13.jar
        Problem:  file is missing
    File: /usr/lib/ambari-metrics-kafka-sink/lib/jackson-mapper-asl-1.9.13.jar
        Problem:  file is missing
    File: /usr/lib/ambari-metrics-kafka-sink/lib/jackson-xc-1.9.13.jar
        Problem:  file is missing
    File: /usr/lib/flume/lib
        Problem:  file is missing
    File: /usr/lib/flume/lib/ambari-metrics-flume-sink-with-common-2.2.2.0.460.jar
        Problem:  file is missing
    File: /usr/lib/storm/lib
        Problem:  file is missing
    File: /usr/lib/storm/lib/ambari-metrics-storm-sink-with-common-2.2.2.0.460.jar
        Problem:  file is missing
Highlighted

Re: RPM missing files during install

Mentor

@KMan

Have you double checked your

# /etc/yum.repos.d

There could be inconsistencies in your repo

Highlighted

Re: RPM missing files during install

I spent all day today cleaning up the repos, disabling any unnecessary repos. Uninstalling every component and reinstalling the whole cluster multiple times. No dice. It's the same components that fail repeatedly and it's the same process to fix them. Reinstall the same RPM's again and restart the service.

Yarn Nodemanagers fail to start

SmartSense HST Agent fail to start

Kafka Brokers fail to start.

Metrics Collector fails to start.

If I don't have a solution I might just have to write a script to reinstall the RPM's cause I can't rely on Ambari.

Edit: Adding the fact that all the files are missing are jar files. In case it helps.

Ex of RPM missing files. Is the ambari agent deleting files once they are installed?

Preinstalling the rpm's before I setup HDP on the cluster results in no errors.

[root@<host> /]# yum info spark_2_4_2_0_258-yarn-shuffle.noarch
Loaded plugins: fastestmirror, langpacks, verify
Loading mirror speeds from cached hostfile
Installed Packages
Name        : spark_2_4_2_0_258-yarn-shuffle
Arch        : noarch
Version     : 1.6.1.2.4.2.0
Release     : 258.el6
Size        : 6.6 M
Repo        : installed
From repo   : HDP-2.4
Summary     : Spark Yarn Shuffle
URL         : http://spark.apache.org/
License     : ASL 2.0
Description : Spark Yarn Shuffle Jar
[root@st1-bdp-hadoop-03 /]# yum verify spark_2_4_2_0_258-yarn-shuffle.noarch
Loaded plugins: fastestmirror, langpacks, verify
==================== Installed Packages ====================
spark_2_4_2_0_258-yarn-shuffle.noarch : Spark Yarn Shuffle
    File: /usr/hdp/2.4.2.0-258/hadoop/lib/spark-yarn-shuffle.jar
        Problem:  file is missing
    File: /usr/hdp/2.4.2.0-258/spark/lib/spark-1.6.1.2.4.2.0-258-yarn-shuffle.jar
        Problem:  file is missing
verify done

[root@<host> ambari-agent]# yum verify smartsense-hst.x86_64
Loaded plugins: fastestmirror, langpacks, verify
==================== Installed Packages ====================
smartsense-hst.x86_64 : Hortonworks SmartSense Tool
    File: /etc/hst/conf/anonymization_rules.json
        Problem:  checksum does not match
        Current:  sha256:0ead985082991351d6fc9f7b1ed7243a06d13f549f63584d23652961dce9d0a1
        Original: sha256:9d5402ba85c07665b0b1315661866437804256a2b868688d5e0aaad6e2520752
                                   --------
        Problem:  size does not match
        Current:  882
        Original: 877
                                   --------
        Problem:  mode does not match
        Current:  user:wr-, group:-r-, other:-r-
        Original: user:wrx, group:-rx, other:-rx
                                   --------
        Problem:  mtime does not match
        Current:  Thu Jan 19 21:53:36 2017 (259 days, 3:29:14 later)
        Original: Thu May  5 18:24:22 2016
    File: /etc/hst/conf/capture_levels.json
        Problem:  checksum does not match
        Current:  sha256:9d0afaff37419601af7f7629a001369512dc00d53066249e6c91200cae07a04b
        Original: sha256:5e51a1cdd967eeae34474e7fbebf16ecbea7e4623bf66fe56d2655d9fb1b4555
                                   --------
        Problem:  size does not match
        Current:  632
        Original: 642
                                   --------
        Problem:  mode does not match
        Current:  user:wr-, group:-r-, other:-r-
        Original: user:wrx, group:-rx, other:-rx
                                   --------
        Problem:  mtime does not match
        Current:  Thu Jan 19 21:53:36 2017 (259 days, 3:29:14 later)
        Original: Thu May  5 18:24:22 2016
    File: /etc/hst/conf/hst-agent.ini
        Problem:  checksum does not match
        Current:  sha256:839b03625393b77cdd9fdf0edd167c305b2245c7522ff476381aad1abfd5c799
        Original: sha256:b7c5113089ef60a9e36dfb2645902ccc26de810fd927b699dcb94b5b098514cb
                                   --------
        Problem:  size does not match
        Current:  3.9 k
        Original: 6.0 k
                                   --------
        Problem:  mode does not match
        Current:  user:wr-, group:-r-, other:-r-
        Original: user:wrx, group:-rx, other:-rx
                                   --------
        Problem:  mtime does not match
        Current:  Mon Jan 23 02:09:59 2017 (262 days, 7:45:37 later)
        Original: Thu May  5 18:24:22 2016
    File: /etc/hst/conf/log4j.properties
        Problem:  checksum does not match
        Current:  sha256:6b49a2e1df7aa81ba3ff9f10fd4794fe5f9c78ea683c95c66ec1ff7cf7dea6ec
        Original: sha256:d0d45c50bb5876ef6dfd3f2ebd14444ab2037e5d177cfd13e7eb897afa43a4c4
                                   --------
        Problem:  size does not match
        Current:          1450 B
        Original:         1440 B
                                   --------
        Problem:  mode does not match
        Current:  user:wr-, group:-r-, other:-r-
        Original: user:wrx, group:-rx, other:-rx
                                   --------
        Problem:  mtime does not match
        Current:  Thu Jan 19 21:53:36 2017 (259 days, 3:29:14 later)
        Original: Thu May  5 18:24:22 2016
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/alerts.json
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/configuration
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/configuration/anonymization-rules.xml
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/configuration/capture-levels.xml
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/configuration/hst-agent-conf.xml
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/configuration/hst-common-conf.xml
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/configuration/hst-log4j.xml
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/configuration/hst-server-conf.xml
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/metainfo.xml
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/files
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/files/.gitignore
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/files/rpm
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/files/rpm/.gitignore
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/files/view
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/files/view/.gitignore
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/files/view/smartsense-ambari-view-1.2.2.0-460.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/files/view/view_default.xml
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/files/view/view_instance.xml
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/scripts
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/scripts/hst.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/scripts/hst_agent.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/scripts/hst_script.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/scripts/hst_server.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/scripts/hst_service.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/scripts/params.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/scripts/service_check.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/package/scripts/utils.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/role_command_order.json
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/themes
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/themes/theme.json
        Problem:  file is missing
    File: /usr/hdp/share/hst/ambari-service/SMARTSENSE/themes/theme_v2.json
        Problem:  file is missing
    File: /usr/hdp/share/hst/bin
        Problem:  file is missing
    File: /usr/hdp/share/hst/bin/hst-env.sh
        Problem:  file is missing
    File: /usr/hdp/share/hst/bin/hst-scheduled-capture.sh
        Problem:  file is missing
    File: /usr/hdp/share/hst/bin/install-helper.sh
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/Bootstrap.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/Bootstrap.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/Controller.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/Controller.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/HSTAgent.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/HSTConfig.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/HSTConfig.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/PropertyReader.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/ServerAPI.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/ServerAPI.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/__init__.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/__init__.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/anonymize.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/AgentUpgradeCommand.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/AnonymizeBundleCommand.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/ArchiveDataCommand.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/CancelRequestCommand.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/CleanupCommand.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/CollectDataCommand.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/CollectDataCommand.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/DownloadScriptCommand.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/GetScriptsCommand.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/UpdateConfigCommand.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/UploadCommand.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/__init__.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/command/__init__.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/commands.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/commands.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/cron.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/heartbeat.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/heartbeat.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/hostname.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/levels.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/levels.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/lock.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/lock.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/main.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/oscheck.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/oscheck.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/patch.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/request.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/script.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/script.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/security.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/security.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/shell.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/shell.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/upgrade
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/upgrade/RequestWithMethod.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/upgrade/UpgradeConfig120.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/upgrade/UpgradeService120.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/upgrade/__init__.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/upgrade/upgrade.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/util.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/lib/hst_agent/util.pyc
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-agent/python-wrap
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/activation-1.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/ambari-views-1.7.0.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/anonymization-1.2.2.0-460.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/aopalliance-1.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/asm-3.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/asm-3.3.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/avalon-framework-4.1.3.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/cglib-2.2.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/cglib-nodep-2.2.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/com.sun.el-1.0.0.v201105211818.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commonj.sdo-2.1.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-1.2.2.0-460.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-beanutils-1.7.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-beanutils-core-1.8.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-cli-1.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-codec-1.9.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-collections-3.2.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-compress-1.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-configuration-1.6.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-digester-1.8.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-io-2.4.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-lang-2.6.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-lang3-3.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-lang3-3.3.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-logging-1.1.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-logging-1.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-net-1.4.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/commons-vfs2-2.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/cron-parser-core-2.8.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/cron-utils-3.1.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/derby-10.9.1.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/easymock-3.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/eclipselink-2.4.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/gatewayClient-1.2.2.0-460.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/gson-2.2.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/guava-14.0.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/guice-3.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/guice-assistedinject-3.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/guice-multibindings-3.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/guice-persist-3.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/guice-servlet-3.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/hamcrest-core-1.3.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/htime-1.0.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/httpclient-4.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/httpcore-4.4.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/httpmime-4.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jackson-core-asl-1.9.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jackson-jaxrs-1.9.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jackson-mapper-asl-1.9.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jackson-xc-1.9.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/javassist-3.17.1-GA.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/javax.el-2.1.0.v201105211819.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/javax.inject-1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/javax.persistence-2.0.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/javax.servlet-2.5.0.v201103041518.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/javax.servlet.jsp-2.1.0.v201105211820.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/javax.servlet.jsp.jstl-1.2.0.v201105211821.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jaxb-api-2.2.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jaxb-impl-2.2.3-1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jersey-client-1.11.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jersey-core-1.11.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jersey-guice-1.11.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jersey-json-1.11.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jersey-multipart-1.11.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jersey-server-1.11.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jersey-servlet-1.11.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jettison-1.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-client-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-continuation-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-http-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-io-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-jsp-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-security-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-server-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-servlet-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-servlets-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-util-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-webapp-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jetty-xml-7.6.7.v20120910.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/joda-time-2.3.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jsch-0.1.52.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jsoup-1.8.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jsp-api-2.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jsr250-api-1.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/jsr305-1.3.9.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/junit-4.11.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/log4j-1.2.17.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/logkit-1.0.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/mailapi-1.5.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/maven-scm-api-1.4.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/maven-scm-provider-svn-commons-1.4.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/maven-scm-provider-svnexe-1.4.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/mimepull-1.6.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/objenesis-1.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/objenesis-tck-1.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/org.apache.jasper.glassfish-2.1.0.v201110031002.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/org.apache.taglibs.standard.glassfish-1.2.0.v201112081803.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/org.eclipse.jdt.core-3.7.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/oro-2.0.8.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/plexus-utils-1.5.6.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/powermock-api-easymock-1.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/powermock-api-support-1.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/powermock-core-1.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/powermock-module-junit4-1.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/powermock-module-junit4-common-1.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/powermock-reflect-1.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/regexp-1.3.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/servlet-api-2.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/slf4j-api-1.7.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/slf4j-jdk14-1.7.7.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/slf4j-log4j12-1.7.5.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/smartsense-gateway-1.2.2.0-460.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/smartsense-hst-1.2.2.0-460.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/smtp-1.5.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-aop-3.0.7.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-asm-3.0.7.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-beans-3.0.7.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-context-3.0.7.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-core-3.0.7.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-expression-3.0.7.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-jdbc-3.0.7.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-security-config-3.1.2.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-security-core-3.1.2.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-security-web-3.1.2.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-tx-3.0.7.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/spring-web-3.0.7.RELEASE.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/stateless4j-2.5.0.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/stax-api-1.0-2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/stax-api-1.0.1.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-common/lib/xz-1.2.jar
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-gateway/python-wrap
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/WEB-INF
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets/hstapp-0d8dee290917e84534d554e3393e2598.css
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets/hstapp-5db66ede969d324e420f4053a0139e13.js
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets/images
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets/images/hortonworks-logo-555555-cb831a5b1cf99fd004f671962de2798a.png
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets/images/hortonworks-logo-79c433-e25f943124d2e451123d2c930a09ede1.png
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets/images/hortonworks-logo-9d9d9d-0756bc37e8a7cbebb3d93872c1b3bf28.png
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets/images/hortonworks-logo-ffffff-e2c63b1388ee67626d284d0a070780ad.png
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets/images/hortonworks-logo-green-d3847d21b50b9afdd214edccdfaf2f79.png
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets/images/hortonworks-logo-wht-1082af8aadaefc935ffca99cb96bb2a8.png
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets/vendor-644a1cfbdf04d5c5c317d4de5f3b18f5.js
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/assets/vendor-6889eac3f8cd620d9f9066ac4297e9ca.css
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/crossdomain.xml
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/docs
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/docs/LICENSE
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/favicon.ico
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/fonts
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/fonts/bootstrap
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/fonts/bootstrap/glyphicons-halflings-regular.eot
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/fonts/bootstrap/glyphicons-halflings-regular.svg
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/fonts/bootstrap/glyphicons-halflings-regular.ttf
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/fonts/bootstrap/glyphicons-halflings-regular.woff
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/fonts/bootstrap/glyphicons-halflings-regular.woff2
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/index.html
        Problem:  file is missing
    File: /usr/hdp/share/hst/hst-server/web/robots.txt
        Problem:  file is missing
    File: /usr/hdp/share/hst/parallel-sh.py
        Problem:  file is missing
    File: /usr/hdp/share/hst/patches
        Problem:  file is missing
    File: /usr/hdp/share/hst/patches/hdp-select-patchST37.sh
        Problem:  file is missing
    File: /var/run/hst
        Problem:  file is missing
verify done
Highlighted

Re: RPM missing files during install

Mentor

@KMan

Dont dispair we all went through that. what document are you following? I just guess you are doing the right thing. Let do it one more time do a clean OS install no java packages.!

I have attached a file here with the install steps for HDP 2.4 RH/CentOS 6.8. Please just follow it until you launch the Ambari wizard.

This was a single node installation,incase of multi node

[root@london ~]# ssh-keygen 
[root@london ~]# chmod 700 ~/.ssh
[root@london]# cat ~/.ssh/id_rsa.pub >> authorized_keys 
[root@london]# chmod 600 ~/.ssh/authorized_keys

Copy the authorized_keys to all the other cluster nodes /.ssh/

It should work out on the ambari repo make sure you choose the correct public repository and OS version. When configuraing the databases for oozie ,hive remember to set the NameNode Java heap size under HDFS->Summary ->settings to more than 4 GB RAM ! Mahout,Ranger;ranger KMS ,SmartSense,Accumulo or HBASE service can be added later,

Please let me know Happy Hadooping !

Highlighted

Re: RPM missing files during install

Some background info. I am not doing these things manually. I have created Ansible deployment scripts that deploy the whole cluster using Ambari blueprints and MySQL HA. Hive and Oozie both talk to MySQL. I also have an uninstallation script that cleans up any HDP/Ambari footprints on the servers. It drops the relevant MySQL DB's as well. In 15-20 mins you have an 8 node cluster up and running from scratch.

These nodes do not have internet connectivity and use an internal repo setup for my team.

For now I am modifying my installation scripts to install the 4 RPM's before Ambari kicks off the Service install. The same RPM's failing to install every time gives me the impression its some dependency issue. The folders or permissions arent setup when these RPM's are being attempted to install and they are fixed further down which causes the rest of the RPM's to install without any issues. I would have resolved this issue If I had Ambari logs that captured the output of the RPM installs.

My hostgroup setup is posted below. I use these to generate the blueprint.

host_groups:
- name: Master_Nodes
  cardinality: '1'
  components:
  - name: SPARK_CLIENT
  - name: YARN_CLIENT
  - name: HDFS_CLIENT
  - name: SQOOP
  - name: METRICS_MONITOR
  - name: HIVE_METASTORE
  - name: NAMENODE
  - name: TEZ_CLIENT
  - name: ZOOKEEPER_CLIENT
  - name: HCAT
  - name: PIG
  - name: HST_AGENT
  - name: MAPREDUCE2_CLIENT
  - name: OOZIE_CLIENT
  - name: HBASE_CLIENT
  - name: HIVE_CLIENT
- name: Secondary_Master_Nodes
  cardinality: '1'
  components:
  - name: AMBARI_SERVER
  - name: HIVE_SERVER
  - name: METRICS_MONITOR
  - name: HBASE_MASTER
  - name: TEZ_CLIENT
  - name: ZOOKEEPER_CLIENT
  - name: HCAT
  - name: WEBHCAT_SERVER
  - name: SECONDARY_NAMENODE
  - name: HST_AGENT
  - name: OOZIE_CLIENT
  - name: METRICS_COLLECTOR
  - name: METRICS_GRAFANA
  - name: SPARK_CLIENT
  - name: YARN_CLIENT
  - name: HDFS_CLIENT
  - name: SQOOP
  - name: HST_SERVER
  - name: HISTORYSERVER
  - name: PIG
  - name: MAPREDUCE2_CLIENT
  - name: HBASE_CLIENT
  - name: SPARK_JOBHISTORYSERVER
  - name: OOZIE_SERVER
  - name: APP_TIMELINE_SERVER
  - name: HIVE_CLIENT
  - name: RESOURCEMANAGER
- name: Kafka_Brokers
  cardinality: '3'
  components:
  - name: KAFKA_BROKER
  - name: HST_AGENT
  - name: ZOOKEEPER_SERVER
  - name: METRICS_MONITOR
- name: Slave_Nodes
  cardinality: '3'
  components:
  - name: NODEMANAGER
  - name: SPARK_CLIENT
  - name: YARN_CLIENT
  - name: HDFS_CLIENT
  - name: SQOOP
  - name: METRICS_MONITOR
  - name: TEZ_CLIENT
  - name: ZOOKEEPER_CLIENT
  - name: HCAT
  - name: PIG
  - name: HST_AGENT
  - name: MAPREDUCE2_CLIENT
  - name: OOZIE_CLIENT
  - name: HBASE_REGIONSERVER
  - name: DATANODE
  - name: HBASE_CLIENT
  - name: HIVE_CLIENT
Highlighted

Re: RPM missing files during install

Mentor

@KMan

For sure the ansible scripts incorporate all the steps I included in the txt file for you. Its good to give complete info on your setup whether single/multi,versions and all relevant info but of course removing sensitive data that help you get a quick and accurate help & info,I see I wasted alot of time because of the scanty info. Nevertheless cany ou share your ansible script?

- Are you going through a proxy or firewallto the internal repos ?
- The Ambari server logs should log all the errors,so don't ignore it  can you copy&paste you last  log?
- The dependency issue is already taken car of in the tarball on the webserver because you can install them manually!
- Can you make sure all the 8 nodes in your cluster OS RPM's/patches are at the same level and can access the internal repo webserver if you have pdsh installed you can quickly compare the nodes rpm status.

Cheers

Don't have an account?
Coming from Hortonworks? Activate your account here