Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Need help with Ambari

Highlighted

Need help with Ambari

I am installing Hadoop using Ambari .

I got error during spark client install .

here is the error

Fail: Execution of '/usr/bin/yum -d 0 -e 0 -y install 'spark_2_2_*'' returned 1. Error: Nothing to do
2016-09-13 16:35:38,220 - Could not determine HDP version for component spark-client by calling '/usr/bin/hdp-select status spark-client > /tmp/tmpJxg9F3'. Return Code: 1, Output: ERROR: Invalid package - spark-client

Packages:
  accumulo-client
  accumulo-gc
  accumulo-master
  accumulo-monitor
  accumulo-tablet
  accumulo-tracer
  falcon-client
  falcon-server
  flume-server
  hadoop-client
  hadoop-hdfs-datanode
  hadoop-hdfs-journalnode
  hadoop-hdfs-namenode
  hadoop-hdfs-nfs3
  hadoop-hdfs-portmap
  hadoop-hdfs-secondarynamenode
  hadoop-mapreduce-historyserver
  hadoop-yarn-nodemanager
  hadoop-yarn-resourcemanager
  hadoop-yarn-timelineserver
  hbase-client
  hbase-master
  hbase-regionserver
  hive-metastore
  hive-server2
  hive-webhcat
  kafka-broker
  knox-server
  mahout-client
  oozie-client
  oozie-server
  phoenix-client
  ranger-admin
  ranger-usersync
  slider-client
  sqoop-client
  sqoop-server
  storm-client
  storm-nimbus
  storm-slider-client
  storm-supervisor
  zookeeper-client
  zookeeper-server
Aliases:
  accumulo-server
  all
  client
  hadoop-hdfs-server
  hadoop-mapreduce-server
  hadoop-yarn-server
  hive-server
.
9 REPLIES 9
Highlighted

Re: Need help with Ambari

Try this: (Please replace the version 2.2.x.x.x with your version number there)

rm -r /usr/hdp/current/spark-client     

ln -s /usr/hdp/2.2.x.x.x/spark /usr/hdp/current/spark-client

So it should look like:

ls -l  /usr/hdp/current/spark-client

lrwxrwxrwx. 1 root root 26 Sep 14 13:04 /usr/hdp/current/spark-client -> /usr/hdp/2.2.x.x.x/spark
Highlighted

Re: Need help with Ambari

Rising Star

That error message indicates the spark package is not available.

Are you adding Spark to an existing installation, or setting up a new cluster? What version of HDP are you using?

Please try running /usr/bin/hdp-select status spark-client and let me know what it returns.

Highlighted

Re: Need help with Ambari

[root@pppdc9prdc8u hdp]# /usr/bin/hdp-select status spark-client ERROR: Invalid package - spark-client

Highlighted

Re: Need help with Ambari

Rising Star

Are you adding Spark or setting up a new cluster? If possible can you complete install without Spark and add individually? Also, what version of HDP are you installing/using?

Highlighted

Re: Need help with Ambari

Thanks Ben for reply .

New setup .By the way i found the issue .sprak will work with HDP 2.4 .I excluded the spark and install rest of the services.Installation completed successfully

Now when ambari is trying to start datanode . it failed with error .

Fail: Execution of 'ambari-sudo.sh su hdfs -l -s /bin/bash -c 'ulimit -c unlimited ; /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh --config /etc/hadoop/conf start datanode'' returned 1. -bash: line 0: ulimit: core file size: cannot modify limit: Operation not permitted starting datanode, logging to /var/log/hadoop/hdfs/hadoop-hdfs-datanode-pppdc9prdc96.out

Please suggest

Jas

Highlighted

Re: Need help with Ambari

Rising Star

As this issue at the surface doesn't seem to be related, you may want to close this question and ask a new question as you will get more help.

In the meantime, to get started on the troubleshooting please run the following and post the results.

grep -v ^# /etc/security/limits.conf
Highlighted

Re: Need help with Ambari

Contributor

@Jaspreet Singh Can you please provide below info:

1. ls -ltr /usr/hdp/
2. ls -ltr /usr/hdp/current/
3. yum info spark-client
4. Run below command manually on host and upload the output:
4.1 yum clean all
4.2 /usr/bin/yum -d 0 -e 0 -y install 'spark_2_2_*'

Please let me know how it goes.

Highlighted

Re: Need help with Ambari

Hi lraheja

Thanks for you reply .

here is the result

[root@pppdc9prdc8u hdp]# yum info spark-client Loaded plugins: security HDP-2.2 | 2.9 kB 00:00 HDP-UTILS-1.1.0.20 | 2.9 kB 00:00 Updates-ambari-2.0.0 | 2.9 kB 00:00 Error: No matching Packages to list [root@pppdc9prdc8u hdp]# yum clean all Loaded plugins: security Cleaning repos: HDP-2.2 HDP-UTILS-1.1.0.20 Updates-ambari-2.0.0 rhel_current saltstack Cleaning up Everything [root@pppdc9prdc8u hdp]# /usr/bin/yum -d 0 -e 0 -y install 'spark_2_2_*'

Highlighted

Re: Need help with Ambari

Thanks jk for your quick response . Really appropriate !

still the same issue

this is what i have done

[root@pppdc9prdc8u ~]# rm -r /usr/hdp/current/spark-client rm: cannot remove `/usr/hdp/current/spark-client': No such file or directory [root@pppdc9prdc8u ~]# cd /usr/hdp/ [root@pppdc9prdc8u hdp]# ls 2.2.0.0-2041 current [root@pppdc9prdc8u hdp]# ln -s /usr/hdp/2.2.0.0-2041/spark /usr/hdp/current/spark-client [root@pppdc9prdc8u hdp]# ls -l /usr/hdp/current/spark-client lrwxrwxrwx 1 root root 27 Sep 14 11:26 /usr/hdp/current/spark-client -> /usr/hdp/2.2.0.0-2041/spark

Don't have an account?
Coming from Hortonworks? Activate your account here