404 status code received on PUT method for API: /api/v1/clusters/Hadoop_UAT/services/SPARKCONTROLLER

[#1]

Hi Experts,

We are using following products.

HDP 3.1.5

We have integration with SAP HANA for which SAP HANA spark controller service is installed in ambari.

The service however always show stopped and whenever I try to start it manually from ambari I am getting the following error.

"404 status code received on PUT method for API: /api/v1/clusters/Hadoop_UAT/services/SPARKCONTROLLER"

"Error message: The specified resource doesn't exist: Service not found, clusterName=Hadoop_UAT, serviceName=SPARKCONTROLLER"

However when we go from the host and from the list of services from the host try to start it the service get started on OS level however from ambari it still shows as stopped.

So basically ambari is not able to detect the state of the service.

How to solve the issue any idea.

Thanks and Regards,

Ananya

[#2]

Hi Experts,

Please help here.

Thanks and Regards,

Ananya

[#3]

@Ananya_Misra  I am getting the same problem , did you able to resolve it. If yes pls help me in this . 

Thanks 

Rakesh

[#4]

@Ananya_Misra & @Rakesh99  you might need to clear the and reset the server state once. 

Ambari server: 
ambari-server stop 
ambari-server reset 
ambari-server start 

In past I have seen this issue with wrong java path. But You should see the stderr logs for more details that should reflect the issue.  

[#5]

Hi @GangWar ,

I have one doubt if I run "ambari-server reset" I am getting the following.

"This will remove all cluster host and configuration information from the database. You will be required to re-configure the Ambari server and re-run the cluster wizard."

 Is it safe to run this and also what are the poststeps that need to be performed.

Also you mentioned I can see more details on this in stderr logs. Can you please guide me to the path of the stderr logs.

Thanks and Regards,

Ananya

[#6]

When i try to start hanas manually by going through /usr/sap/spark/controller/bin and start hanas getting below error. 

Current Class path is: /usr/sap/spark/controller/conf:/usr/hdp/3.1.4.0-315/spark2/jars::mysql-connector-java-8.0.20.jar:mysql-connector-java-8.0.20.jar:/usr/hdp/3.1.4.0-315/tez/*:/usr/hdp/3.1.4.0-315/tez/lib/*:/usr/hdp/3.1.4.0-315/tez/conf:/usr/hdp/3.1.4.0-315/hadoop/conf:/etc/hive/conf:/usr/sap/spark/controller/*:/usr/sap/spark/controller/lib/*:/usr/sap/spark/controller/lib/other/*:/usr/sap/spark/controller/lib/hana/*:/usr/hdp/3.1.4.0-315/hadoop/*:/usr/hdp/3.1.4.0-315/hadoop/lib/*:/usr/hdp/3.1.4.0-315/hadoop-hdfs/*:/usr/hdp/3.1.4.0-315/hadoop-hdfs/lib/*
INFO: Starting in Background
INFO: Started in Background.. Please check log file /var/log/hanaes/hana_controller.log for latest status
hanaes@ghdmasterqas01:/usr/sap/spark/controller/bin$ tail -100 /var/log/hanaes/hana_controller.log
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:650)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:632)
Caused by: java.lang.ClassNotFoundException: scala.Function0
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 7 more
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: scala/Function0
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:650)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:632)
Caused by: java.lang.ClassNotFoundException: scala.Function0
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 7 more
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: scala/Function0
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)

[#7]

any update on this problem ? If anyone finds a solution to this problem pls let me know   

[#8]

This seems a problem with installation itself, I would suggest to reinstall the software. 

Caused by: java.lang.ClassNotFoundException: scala.Function0
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 7 more
Error: A JNI error has occurred, please check your installation and try again