Support Questions
Find answers, ask questions, and share your expertise

In lab 4, issue of importing spark.sql

Highlighted

In lab 4, issue of importing spark.sql

Explorer

%spark

importorg.apache.spark.sql.hive.orc._

importorg.apache.spark.sql._

the error message:

java.net.ConnectException: Connection refused (Connection refused)

at java.net.PlainSocketImpl.socketConnect(Native Method)

........

any ideas about this error? Thanks!

10 REPLIES 10
Highlighted

Re: In lab 4, issue of importing spark.sql

Contributor

@Stwo X

Are you running this on a Sandbox. Can you check for any network issues and all HDP services are running.

Also check for exception in zeppelin-interpreter-spark. log .

Highlighted

Re: In lab 4, issue of importing spark.sql

Explorer

yes, Sandbox. It is a host and VM model on the same machine, shouldn't be network issues. Just checked the services, I don't know which services are required. I just followed the guide instruction. Livy, Spark, Spark2, Hive and Zeppelin are running. any other idea?

Highlighted

Re: In lab 4, issue of importing spark.sql

Contributor

@Stwo X check for exception in zeppelin-interpreter-spark. log .

Highlighted

Re: In lab 4, issue of importing spark.sql

Explorer

java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.thrift.transport.TSocket.open(TSocket.java:182) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37) at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:189) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:173) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:338) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:262) at org.apache.zeppelin.scheduler.Job.run(Job.java:176) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

Highlighted

Re: In lab 4, issue of importing spark.sql

Contributor

@Stwo X Do you which server or service is it trying to make a connection . Does it give any other information besides this.

Also check below Apache JIRA

https://issues.apache.org/jira/browse/ZEPPELIN-305

Highlighted

Re: In lab 4, issue of importing spark.sql

Explorer

this is the output. no other information. where to find zeppelin-interpreter-spark.log? i am new to HDP. thanks!

Highlighted

Re: In lab 4, issue of importing spark.sql

Explorer

couldn't locate the zeppelin-interpreter-spark.log.

if is use the built-in web client (http://host:4200) i can successfully execute the spark sql import command.

Highlighted

Re: In lab 4, issue of importing spark.sql

Explorer

the default Spark interpreter property setting, anythong wrong with it?

Properties
namevalue
args
masteryarn-client
spark.app.nameZeppelin
spark.cores.max
spark.executor.memory512m
spark.yarn.keytab
spark.yarn.principal
zeppelin.dep.additionalRemoteRepositoryspark-packages,http://dl.bintray.com/spark-packages/maven,false;
zeppelin.dep.localrepolocal-repo
zeppelin.pyspark.pythonpython
zeppelin.spark.concurrentSQLfalse
zeppelin.spark.importImplicittrue
zeppelin.spark.maxResult
zeppelin.spark.printREPLOutputtrue
zeppelin.spark.sql.stacktracefalse
zeppelin.spark.useHiveContexttrue
Highlighted

Re: In lab 4, issue of importing spark.sql

Explorer

hi prsingh, thanks for your hints.

by configuring the spark interpreter properties:

zeppelin.spark.concurrentSQLtrue
zeppelin.spark.sql.stacktracetrue

the sql import command passed.