Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

In lab 4, issue of importing spark.sql

In lab 4, issue of importing spark.sql

New Contributor

%spark

importorg.apache.spark.sql.hive.orc._

importorg.apache.spark.sql._

the error message:

java.net.ConnectException: Connection refused (Connection refused)

at java.net.PlainSocketImpl.socketConnect(Native Method)

........

any ideas about this error? Thanks!

10 REPLIES 10

Re: In lab 4, issue of importing spark.sql

Contributor

@Stwo X

Are you running this on a Sandbox. Can you check for any network issues and all HDP services are running.

Also check for exception in zeppelin-interpreter-spark. log .

Re: In lab 4, issue of importing spark.sql

New Contributor

yes, Sandbox. It is a host and VM model on the same machine, shouldn't be network issues. Just checked the services, I don't know which services are required. I just followed the guide instruction. Livy, Spark, Spark2, Hive and Zeppelin are running. any other idea?

Highlighted

Re: In lab 4, issue of importing spark.sql

Contributor

@Stwo X check for exception in zeppelin-interpreter-spark. log .

Re: In lab 4, issue of importing spark.sql

New Contributor

java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.thrift.transport.TSocket.open(TSocket.java:182) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37) at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:189) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:173) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:338) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:262) at org.apache.zeppelin.scheduler.Job.run(Job.java:176) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

Re: In lab 4, issue of importing spark.sql

Contributor

@Stwo X Do you which server or service is it trying to make a connection . Does it give any other information besides this.

Also check below Apache JIRA

https://issues.apache.org/jira/browse/ZEPPELIN-305

Re: In lab 4, issue of importing spark.sql

New Contributor

this is the output. no other information. where to find zeppelin-interpreter-spark.log? i am new to HDP. thanks!

Re: In lab 4, issue of importing spark.sql

New Contributor

couldn't locate the zeppelin-interpreter-spark.log.

if is use the built-in web client (http://host:4200) i can successfully execute the spark sql import command.

Re: In lab 4, issue of importing spark.sql

New Contributor

the default Spark interpreter property setting, anythong wrong with it?

Properties
namevalue
args
masteryarn-client
spark.app.nameZeppelin
spark.cores.max
spark.executor.memory512m
spark.yarn.keytab
spark.yarn.principal
zeppelin.dep.additionalRemoteRepositoryspark-packages,http://dl.bintray.com/spark-packages/maven,false;
zeppelin.dep.localrepolocal-repo
zeppelin.pyspark.pythonpython
zeppelin.spark.concurrentSQLfalse
zeppelin.spark.importImplicittrue
zeppelin.spark.maxResult
zeppelin.spark.printREPLOutputtrue
zeppelin.spark.sql.stacktracefalse
zeppelin.spark.useHiveContexttrue

Re: In lab 4, issue of importing spark.sql

New Contributor

hi prsingh, thanks for your hints.

by configuring the spark interpreter properties:

zeppelin.spark.concurrentSQLtrue
zeppelin.spark.sql.stacktracetrue

the sql import command passed.