Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Problem setting up Hue Spark notebook via livy server

Solved Go to solution
Highlighted

Problem setting up Hue Spark notebook via livy server

Contributor

Hello guys, I am trying to setup livy_server for the spark notebooks to work with Hue 3.9.0 with CDH 5.6.0. I am facing the following error:

 

livy server is failing to start with hue database password error:

 

 

[root@xxxxxx hue]# ./build/env/bin/hue livy_server
Traceback (most recent call last):
  File "./build/env/bin/hue", line 12, in <module>
    load_entry_point('desktop==3.9.0', 'console_scripts', 'hue')()
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/desktop/core/src/desktop/manage_entry.py", line 57, in entry
    execute_from_command_line(sys.argv)
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/__init__.py", line 399, in e                                                            xecute_from_command_line
    utility.execute()
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/__init__.py", line 392, in e                                                            xecute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/__init__.py", line 261, in f                                                            etch_command
    commands = get_commands()
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/core/management/__init__.py", line 107, in g                                                            et_commands
    apps = settings.INSTALLED_APPS
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/conf/__init__.py", line 54, in __getattr__
    self._setup(name)
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/conf/__init__.py", line 49, in _setup
    self._wrapped = Settings(settings_module)
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/conf/__init__.py", line 128, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/build/env/lib/python2.6/site-packages/Django-1.6.10-py2.6.egg/django/utils/importlib.py", line 40, in import_modu                                                            le
    __import__(name)
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/desktop/core/src/desktop/settings.py", line 309, in <module>
    "PASSWORD" : desktop.conf.get_database_password(),
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/desktop/core/src/desktop/conf.py", line 1219, in get_database_password
    password = DATABASE.PASSWORD_SCRIPT.get()
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/desktop/core/src/desktop/lib/conf.py", line 140, in get
    return self.config.get_value(data, present=present, prefix=self.prefix, coerce_type=True)
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/desktop/core/src/desktop/lib/conf.py", line 256, in get_value
    return self._coerce_type(raw_val, prefix)
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/desktop/core/src/desktop/lib/conf.py", line 276, in _coerce_type
    return self.type(raw)
  File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/desktop/core/src/desktop/conf.py", line 66, in coerce_password_from_script
    raise subprocess.CalledProcessError(p.returncode, script)
subprocess.CalledProcessError: Command '/var/run/cloudera-scm-agent/process/4295-hue-HUE_SERVER/altscript.sh sec-8-password' returned non-zero exit status 1

I made sure current hue password set in CM is working fine.

 

Please help here.

Engineer
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Problem setting up Hue Spark notebook via livy server

Contributor

This is now fixed.

Engineer

View solution in original post

7 REPLIES 7
Highlighted

Re: Problem setting up Hue Spark notebook via livy server

New Contributor

Hello, Even im facing the same issue.Could anyone please help .

 

 

Highlighted

Re: Problem setting up Hue Spark notebook via livy server

Contributor

This is now fixed.

Engineer

View solution in original post

Highlighted

Re: Problem setting up Hue Spark notebook via livy server

Community Manager

Congratulations on solving your issue @vganji. Would you mind sharing your solution in case it can assist others?


Cy Jervis, Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:
Community Guidelines
How to use the forum
Highlighted

Re: Problem setting up Hue Spark notebook via livy server

New Contributor
Hi all,
I've enabled spark notebook on Hue 3.9 in my CDH 5.5.2 cluster.
I've started LivyServer in this way:

export SPARK_HOME=/opt/cloudera/parcels/CDH/lib/spark
export PATH=$PATH:$SPARK_HOME/bin
/opt/cloudera/parcels/CDH/lib/hue/build/env/bin/hue livy_server&


Then, in a new PySpark notebook I've put this code:

a = sc.textFile("/user/admin/index_data.csv")
a.count()

and I've got this error:

05:43:58.141 [qtp455119019-16] ERROR c.c.hue.livy.server.SessionServlet$ - internal error
java.lang.IllegalStateException: Session is in state error
at com.cloudera.hue.livy.server.interactive.InteractiveWebSession.ensureRunning(InteractiveWebSession.scala:214) ~[livy-assembly.jar:na]
at com.cloudera.hue.livy.server.interactive.InteractiveWebSession.executeStatement(InteractiveWebSession.scala:73) ~[livy-assembly.jar:na]
at com.cloudera.hue.livy.server.interactive.InteractiveSessionServlet$$anonfun$16.apply(InteractiveSessionServlet.scala:128) ~[livy-assembly.jar:na]
at org.scalatra.ScalatraBase$class.org$scalatra$ScalatraBase$$liftAction(ScalatraBase.scala:270) [livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$invoke$1.apply(ScalatraBase.scala:265) ~[livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$invoke$1.apply(ScalatraBase.scala:265) ~[livy-assembly.jar:na]
at org.scalatra.ApiFormats$class.withRouteMultiParams(ApiFormats.scala:178) ~[livy-assembly.jar:na]
at com.cloudera.hue.livy.server.SessionServlet.withRouteMultiParams(SessionServlet.scala:33) [livy-assembly.jar:na]
at org.scalatra.ScalatraBase$class.invoke(ScalatraBase.scala:264) [livy-assembly.jar:na]
at com.cloudera.hue.livy.server.SessionServlet.org$scalatra$json$JsonSupport$$super$invoke(SessionServlet.scala:33) [livy-assembly.jar:na]
at org.scalatra.json.JsonSupport$$anonfun$invoke$1.apply(JsonSupport.scala:74) ~[livy-assembly.jar:na]
at org.scalatra.json.JsonSupport$$anonfun$invoke$1.apply(JsonSupport.scala:68) ~[livy-assembly.jar:na]
at org.scalatra.ApiFormats$class.withRouteMultiParams(ApiFormats.scala:178) ~[livy-assembly.jar:na]
at com.cloudera.hue.livy.server.SessionServlet.withRouteMultiParams(SessionServlet.scala:33) [livy-assembly.jar:na]
at org.scalatra.json.JsonSupport$class.invoke(JsonSupport.scala:68) ~[livy-assembly.jar:na]
at com.cloudera.hue.livy.server.SessionServlet.invoke(SessionServlet.scala:33) [livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$runRoutes$1$$anonfun$apply$8.apply(ScalatraBase.scala:240) ~[livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$runRoutes$1$$anonfun$apply$8.apply(ScalatraBase.scala:238) ~[livy-assembly.jar:na]
at scala.Option.flatMap(Option.scala:170) ~[livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$runRoutes$1.apply(ScalatraBase.scala:238) ~[livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$runRoutes$1.apply(ScalatraBase.scala:237) ~[livy-assembly.jar:na]
at scala.collection.immutable.Stream.flatMap(Stream.scala:442) ~[livy-assembly.jar:na]
at org.scalatra.ScalatraBase$class.runRoutes(ScalatraBase.scala:237) [livy-assembly.jar:na]
at org.scalatra.ScalatraServlet.runRoutes(ScalatraServlet.scala:49) [livy-assembly.jar:na]
at org.scalatra.ScalatraBase$class.runActions$1(ScalatraBase.scala:163) [livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply$mcV$sp(ScalatraBase.scala:175) ~[livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply(ScalatraBase.scala:175) ~[livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply(ScalatraBase.scala:175) ~[livy-assembly.jar:na]
at org.scalatra.ScalatraBase$class.org$scalatra$ScalatraBase$$cradleHalt(ScalatraBase.scala:193) [livy-assembly.jar:na]
at org.scalatra.ScalatraBase$class.executeRoutes(ScalatraBase.scala:175) [livy-assembly.jar:na]
at org.scalatra.ScalatraServlet.executeRoutes(ScalatraServlet.scala:49) [livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$handle$1.apply$mcV$sp(ScalatraBase.scala:113) [livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$handle$1.apply(ScalatraBase.scala:113) [livy-assembly.jar:na]
at org.scalatra.ScalatraBase$$anonfun$handle$1.apply(ScalatraBase.scala:113) [livy-assembly.jar:na]
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57) [livy-assembly.jar:na]
at org.scalatra.DynamicScope$class.withResponse(DynamicScope.scala:80) [livy-assembly.jar:na]
at org.scalatra.ScalatraServlet.withResponse(ScalatraServlet.scala:49) [livy-assembly.jar:na]
at org.scalatra.DynamicScope$$anonfun$withRequestResponse$1.apply(DynamicScope.scala:60) [livy-assembly.jar:na]
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57) [livy-assembly.jar:na]
at org.scalatra.DynamicScope$class.withRequest(DynamicScope.scala:71) [livy-assembly.jar:na]
at org.scalatra.ScalatraServlet.withRequest(ScalatraServlet.scala:49) [livy-assembly.jar:na]
at org.scalatra.DynamicScope$class.withRequestResponse(DynamicScope.scala:59) [livy-assembly.jar:na]
at org.scalatra.ScalatraServlet.withRequestResponse(ScalatraServlet.scala:49) [livy-assembly.jar:na]
at org.scalatra.ScalatraBase$class.handle(ScalatraBase.scala:111) [livy-assembly.jar:na]
at org.scalatra.ScalatraServlet.org$scalatra$servlet$ServletBase$$super$handle(ScalatraServlet.scala:49) [livy-assembly.jar:na]
at org.scalatra.servlet.ServletBase$class.handle(ServletBase.scala:43) [livy-assembly.jar:na]
at com.cloudera.hue.livy.server.SessionServlet.org$scalatra$MethodOverride$$super$handle(SessionServlet.scala:33) [livy-assembly.jar:na]
at org.scalatra.MethodOverride$class.handle(MethodOverride.scala:28) [livy-assembly.jar:na]
at com.cloudera.hue.livy.server.SessionServlet.handle(SessionServlet.scala:33) [livy-assembly.jar:na]
at org.scalatra.ScalatraServlet.service(ScalatraServlet.scala:54) [livy-assembly.jar:na]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.servlet.ServletHolder.handle(ServletHolder.java:808) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.server.Server.handle(Server.java:497) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.server.HttpChannel.handle(HttpChannel.java:310) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) [livy-assembly.jar:na]
at com.cloudera.hue.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) [livy-assembly.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_95]
127.0.0.1 - - - 31/Mar/2016:05:43:58 -0700 "POST /sessions/0/statements HTTP/1.1" 500 58


Could you help me?
Thanks a lot

Re: Problem setting up Hue Spark notebook via livy server

New Contributor

Vganji,

 

How did you solve it?

 

Thanks,

Bramd

Highlighted

Re: Problem setting up Hue Spark notebook via livy server

Super Guru

In Cloudera Manager 5.5 and higher, service passwords are stored in Cloudera Manager so that they are not exposed in the process directories.  As part of this, the passwords are stored in a jceks file in each process directory.  A password is then passed to the start scripts to decrypt the file containing the passwords.

 

When attempting to run the "hue" command from the command line, you will get the following:

 

File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/desktop/core/src/desktop/lib/conf.py", line 276, in _coerce_type
return self.type(raw)
File "/opt/cloudera/parcels/CDH-5.6.0-1.cdh5.6.0.p0.45/lib/hue/desktop/core/src/desktop/conf.py", line 66, in coerce_password_from_script
raise subprocess.CalledProcessError(p.returncode, script)
subprocess.CalledProcessError: Command '/var/run/cloudera-scm-agent/process/4295-hue-HUE_SERVER/altscript.sh sec-8-password' returned non-zero exit status 1

 

This is because "altscript.sh" was not passed the password from CM.

To help allow you to be able to run commands from the commandline, I suggest trying:

 

# export HUE_CONF_DIR=/var/run/cloudera-scm-agent/process/`ls -lrt /var/run/cloudera-scm-agent/process/ | awk '{print $9}' |grep HUE_SERVER| tail -1`

 

# HUE_IGNORE_PASSWORD_SCRIPT_ERRORS=1 HUE_DATABASE_PASSWORD=password /opt/cloudera/parcels/CDH/lib/hue/build/env/bin/hue livy_server

 

Since the Spark Notebook is not supported in CDH yet, we cannot guarantee it will work as desired, but at the least, the steps I give should help you get around the exception that @vganji mentioned.

Highlighted

Re: Problem setting up Hue Spark notebook via livy server

New Contributor
I was still stuck with not knowing the HUE_DATABASE_PASSWORD

You can get it in cleartext from the CM REST API.

Make sure you're logged in to CM with admin rights.

http://:7180/api/v3/clusters//services//config?view=full

HTH
Bramd
Don't have an account?
Coming from Hortonworks? Activate your account here