Reply
New Contributor
Posts: 5
Registered: ‎09-17-2017
Accepted Solution

Getting Error while connecting spark from Zeppelin

Hi Team,

 

when connecting zeppline from cloudera cluster with spark we are getting the below error. We exported the below variables in the zepplien-env.sh. 

 

 

export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
export SPARK_HOME=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/lib/spark
export ZEPPELIN_MEM=-Xmx5g
export HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/lib/hive
export HADOOP_CONF_DIR=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/

 

 

ERROR, exception: org.apache.zeppelin.interpreter.InterpreterException: org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.trans
port.TTransportException: java.net.ConnectException: Connection refused, result: org.apache.zeppelin.interpreter.InterpreterException: org.apache.thr
ift.transport.TTransportException: java.net.ConnectException: Connection refused
INFO [2018-02-16 13:51:19,844] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1504078509719_113511079 finished by schedu
ler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session814817802
ERROR [2018-02-16 13:51:20,224] ({Thread-57} RemoteInterpreterEventPoller.java[run]:102) - Can't get RemoteInterpreterEvent
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:92)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:100)
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
... 7 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 8 more
ERROR [2018-02-16 13:51:21,224] ({Thread-57} RemoteInterpreterEventPoller.java[run]:102) - Can't get RemoteInterpreterEvent
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)

New Contributor
Posts: 5
Registered: ‎09-17-2017

Re: Getting Error while connecting spark from Zeppelin

Hi All,

 

After adding the below two parameter in the spark interpreter, the issue got resolved.

 

spark.yarn.keytab
spark.yarn.principal

 

Regards

Chaitanya 

Announcements

Currently incubating in Cloudera Labs:

Envelope
HTrace
Ibis
Impyla
Livy
Oryx
Phoenix
Spark Runner for Beam SDK
Time Series for Spark
YCSB