Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

In lab 4, issue of importing spark.sql

Highlighted

In lab 4, issue of importing spark.sql

Explorer

%spark

importorg.apache.spark.sql.hive.orc._

importorg.apache.spark.sql._

the error message:

java.net.ConnectException: Connection refused (Connection refused)

at java.net.PlainSocketImpl.socketConnect(Native Method)

........

any ideas about this error? Thanks!

10 REPLIES 10
Highlighted

Re: In lab 4, issue of importing spark.sql

Contributor

@Stwo X

Are you running this on a Sandbox. Can you check for any network issues and all HDP services are running.

Also check for exception in zeppelin-interpreter-spark. log .

Highlighted

Re: In lab 4, issue of importing spark.sql

Explorer

yes, Sandbox. It is a host and VM model on the same machine, shouldn't be network issues. Just checked the services, I don't know which services are required. I just followed the guide instruction. Livy, Spark, Spark2, Hive and Zeppelin are running. any other idea?

Highlighted

Re: In lab 4, issue of importing spark.sql

Contributor

@Stwo X check for exception in zeppelin-interpreter-spark. log .

Highlighted

Re: In lab 4, issue of importing spark.sql

Explorer

java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.thrift.transport.TSocket.open(TSocket.java:182) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37) at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:189) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:173) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:338) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:262) at org.apache.zeppelin.scheduler.Job.run(Job.java:176) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

Highlighted

Re: In lab 4, issue of importing spark.sql

Contributor

@Stwo X Do you which server or service is it trying to make a connection . Does it give any other information besides this.

Also check below Apache JIRA

https://issues.apache.org/jira/browse/ZEPPELIN-305

Highlighted

Re: In lab 4, issue of importing spark.sql

Explorer

this is the output. no other information. where to find zeppelin-interpreter-spark.log? i am new to HDP. thanks!

Re: In lab 4, issue of importing spark.sql

Explorer

couldn't locate the zeppelin-interpreter-spark.log.

if is use the built-in web client (http://host:4200) i can successfully execute the spark sql import command.

Highlighted

Re: In lab 4, issue of importing spark.sql

Explorer

the default Spark interpreter property setting, anythong wrong with it?

Properties
namevalue
args
masteryarn-client
spark.app.nameZeppelin
spark.cores.max
spark.executor.memory512m
spark.yarn.keytab
spark.yarn.principal
zeppelin.dep.additionalRemoteRepositoryspark-packages,http://dl.bintray.com/spark-packages/maven,false;
zeppelin.dep.localrepolocal-repo
zeppelin.pyspark.pythonpython
zeppelin.spark.concurrentSQLfalse
zeppelin.spark.importImplicittrue
zeppelin.spark.maxResult
zeppelin.spark.printREPLOutputtrue
zeppelin.spark.sql.stacktracefalse
zeppelin.spark.useHiveContexttrue
Highlighted

Re: In lab 4, issue of importing spark.sql

Explorer

hi prsingh, thanks for your hints.

by configuring the spark interpreter properties:

zeppelin.spark.concurrentSQLtrue
zeppelin.spark.sql.stacktracetrue

the sql import command passed.

Don't have an account?
Coming from Hortonworks? Activate your account here