Support Questions
Find answers, ask questions, and share your expertise

Spark Livy 2 server error and not running

Explorer

Hi,

Unable to start the Spark Livy Server 2

ExecutionFailed: Execution of 'curl -s -o /dev/null -w'%{http_code}' --negotiate -u: -k http://random_host.net:8999/sessions | grep 200 ' returned 1. curl: option --negotiate: the installed libcurl version doesn't support this
curl: try 'curl --help' or 'curl --manual' for more information
) 

I have tried to install libcurl . but it has latest package

# yum install libcurl
Loaded plugins: fastestmirror, langpacks
Package matching libcurl-7.29.0-35.el7.centos.x86_64 already installed. Checking for update.
Nothing to do

Ambari Says it is started and running .But the error wont go. Refer image attached.

The port is opened and running in the background:

netstat -plant | grep 8999tcp6      
 0      0 :::8999                 :::*                    LISTEN      4677/java

I have Spark2 livy server in two machines.

1.Machine 1 Spark livy server 2 is running.

2.Machine 2 Spark Livy server 2 is running as per Ambari and terminal . However Ambari says error.

I have verified both the machines logs , it shows both error are similar

(ie Hadoop folder missing , java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset.) n(java.io.FileNotFoundException and File does not exist: /livy2-recovery/v1/batch/st ) the error im getting from 2nd machine not from the 1st machine.

I have disabled the alert and enabled the alert . still it doesn't work.I have ambari agent restarted twice .

. Still it doesn't work.

sp.png

Attached log : sp.txt

1 REPLY 1

Explorer
It is nothing to do with the spark. may be package-mismatch on libcurl 

do a complete uninstall of curl and try a fresh install. 
by 
wget http://curl.haxx.se/download/curl-7.48.0.tar.gz 
tar -xvzf curl-7.48.0.tar.gz 
cd curl-7.48.0
./configure
sudo make
sudo make install

If you are particular about the version, please refer https://curl.haxx.se/download.html .

or

sudo apt-get update
sudo apt-get install curl 
sudo apt-get install libcurl3

please change apt-get according to your Linux flavor