Cloudera Labs
Provide feedback on Cloudera Labs
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Getting Error while connecting spark from Zeppelin

Solved Go to solution

Getting Error while connecting spark from Zeppelin

New Contributor

Hi Team,

 

when connecting zeppline from cloudera cluster with spark we are getting the below error. We exported the below variables in the zepplien-env.sh. 

 

 

export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
export SPARK_HOME=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/lib/spark
export ZEPPELIN_MEM=-Xmx5g
export HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/lib/hive
export HADOOP_CONF_DIR=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/

 

 

ERROR, exception: org.apache.zeppelin.interpreter.InterpreterException: org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.trans
port.TTransportException: java.net.ConnectException: Connection refused, result: org.apache.zeppelin.interpreter.InterpreterException: org.apache.thr
ift.transport.TTransportException: java.net.ConnectException: Connection refused
INFO [2018-02-16 13:51:19,844] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1504078509719_113511079 finished by schedu
ler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session814817802
ERROR [2018-02-16 13:51:20,224] ({Thread-57} RemoteInterpreterEventPoller.java[run]:102) - Can't get RemoteInterpreterEvent
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:92)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:100)
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
... 7 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 8 more
ERROR [2018-02-16 13:51:21,224] ({Thread-57} RemoteInterpreterEventPoller.java[run]:102) - Can't get RemoteInterpreterEvent
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Getting Error while connecting spark from Zeppelin

New Contributor

Hi All,

 

After adding the below two parameter in the spark interpreter, the issue got resolved.

 

spark.yarn.keytab
spark.yarn.principal

 

Regards

Chaitanya 

1 REPLY 1

Re: Getting Error while connecting spark from Zeppelin

New Contributor

Hi All,

 

After adding the below two parameter in the spark interpreter, the issue got resolved.

 

spark.yarn.keytab
spark.yarn.principal

 

Regards

Chaitanya