02-16-2018 05:59 AM
Hi Team,
when connecting zeppline from cloudera cluster with spark we are getting the below error. We exported the below variables in the zepplien-env.sh.
export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
export SPARK_HOME=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/lib/spark
export ZEPPELIN_MEM=-Xmx5g
export HIVE_HOME=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/lib/hive
export HADOOP_CONF_DIR=/opt/cloudera/parcels/CDH-5.8.3-1.cdh5.8.3.p0.2/
ERROR, exception: org.apache.zeppelin.interpreter.InterpreterException: org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.trans
port.TTransportException: java.net.ConnectException: Connection refused, result: org.apache.zeppelin.interpreter.InterpreterException: org.apache.thr
ift.transport.TTransportException: java.net.ConnectException: Connection refused
INFO [2018-02-16 13:51:19,844] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1504078509719_113511079 finished by schedu
ler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session814817802
ERROR [2018-02-16 13:51:20,224] ({Thread-57} RemoteInterpreterEventPoller.java[run]:102) - Can't get RemoteInterpreterEvent
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:92)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:100)
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
... 7 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 8 more
ERROR [2018-02-16 13:51:21,224] ({Thread-57} RemoteInterpreterEventPoller.java[run]:102) - Can't get RemoteInterpreterEvent
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
Solved! Go to Solution.
02-21-2018 07:47 AM
Hi All,
After adding the below two parameter in the spark interpreter, the issue got resolved.
spark.yarn.keytab
spark.yarn.principal
Regards
Chaitanya
Currently incubating in Cloudera Labs:
Envelope