Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error Spark 2 HDP 2.5.3 Ubuntu?

Error Spark 2 HDP 2.5.3 Ubuntu?

New Contributor

Hello I get the following installation error:

2018-11-28 02:15:46,115 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2018-11-28 02:15:46,116 - Group['spark'] {}
2018-11-28 02:15:46,117 - Group['hadoop'] {}
2018-11-28 02:15:46,118 - Group['users'] {}
2018-11-28 02:15:46,118 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2018-11-28 02:15:46,119 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2018-11-28 02:15:46,120 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2018-11-28 02:15:46,120 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2018-11-28 02:15:46,121 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
2018-11-28 02:15:46,122 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
2018-11-28 02:15:46,123 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2018-11-28 02:15:46,123 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2018-11-28 02:15:46,124 - User['hcat'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2018-11-28 02:15:46,125 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2018-11-28 02:15:46,135 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2018-11-28 02:15:46,137 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2018-11-28 02:15:46,143 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2018-11-28 02:15:46,143 - Group['hdfs'] {}
2018-11-28 02:15:46,144 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
2018-11-28 02:15:46,144 - FS Type: 
2018-11-28 02:15:46,145 - Directory['/etc/hadoop'] {'mode': 0755}
2018-11-28 02:15:46,188 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2018-11-28 02:15:46,189 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2018-11-28 02:15:46,225 - Initializing 2 repositories
2018-11-28 02:15:46,226 - Repository['HDP-2.5'] {'base_url': 'http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.5.3.0', 'action': ['create'], 'components': [u'HDP', 'main'], 'repo_template': '{{package_type}} {{base_url}} {{components}}', 'repo_file_name': 'HDP', 'mirror_list': None}
2018-11-28 02:15:46,232 - File['/tmp/tmpw1p5LH'] {'content': 'deb http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.5.3.0 HDP main'}
2018-11-28 02:15:46,233 - Writing File['/tmp/tmpw1p5LH'] because contents don't match
2018-11-28 02:15:46,233 - File['/tmp/tmp2OqNMR'] {'content': StaticFile('/etc/apt/sources.list.d/HDP.list')}
2018-11-28 02:15:46,242 - Writing File['/tmp/tmp2OqNMR'] because contents don't match
2018-11-28 02:15:46,243 - Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu14', 'action': ['create'], 'components': [u'HDP-UTILS', 'main'], 'repo_template': '{{package_type}} {{base_url}} {{components}}', 'repo_file_name': 'HDP-UTILS', 'mirror_list': None}
2018-11-28 02:15:46,244 - File['/tmp/tmpWRGt4b'] {'content': 'deb http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu14 HDP-UTILS main'}
2018-11-28 02:15:46,245 - Writing File['/tmp/tmpWRGt4b'] because contents don't match
2018-11-28 02:15:46,268 - File['/tmp/tmpHcmyrM'] {'content': StaticFile('/etc/apt/sources.list.d/HDP-UTILS.list')}
2018-11-28 02:15:46,268 - Writing File['/tmp/tmpHcmyrM'] because contents don't match
2018-11-28 02:15:46,270 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-11-28 02:15:46,408 - Skipping installation of existing package unzip
2018-11-28 02:15:46,408 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-11-28 02:15:46,542 - Skipping installation of existing package curl
2018-11-28 02:15:46,543 - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-11-28 02:15:46,673 - Skipping installation of existing package hdp-select
2018-11-28 02:15:47,071 - Package['spark2-2-5-3-0-37'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-11-28 02:15:47,276 - Installing package spark2-2-5-3-0-37 ('/usr/bin/apt-get -q -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install spark2-2-5-3-0-37')
Command failed after 1 tries

could somebody please helo me how to solve this, what do I have to pay attention to? How to solve this?

Spark is needed here, the other services run.

1 REPLY 1

Re: Error Spark 2 HDP 2.5.3 Ubuntu?

Expert Contributor

What happens when you run /usr/bin/apt-get -q -o Dpkg::Options::=--force-confdef --allow-unauthenticated --assume-yes install spark2-2-5-3-0-37?