We are not able to connect Spark SQL through JDBC driver. Usually we use 10005 port to connect spark SQL in previous Horton works ( normal). But now we are using Hortonworks data cloud from AWS market place. ( HDP 2.5 and Apache Spark 2.0)
Your doumentation only suggests to connect via spark shell : http://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.14.1/bk_hdcloud-aws/content/spark/i...
but for hive, there are multiple options, like jdbc and odbc : http://docs.hortonworks.com/HDPDocuments/HDCloudAWS/HDCloudAWS-1.14.1/bk_hdcloud-aws/content/hive-jd...
Please help me to connect spark via jdbc from work bench for hortonworks data cloud.
Hello @Aravind J L
With Hortonworks Data Cloud the Spark Thrift Server is not a pre-configured component. You can manually set it up using the following steps:
1. Login to Ambari on the Spark cluster type using your admin credentials
2. Make the config changes required if any to ensure the Spark Thrift Server is sized correctluy and you have adequate capacity on the cluster
3. Select the node on which you want to run the Spark Thrift Server and add it as a component
4. Either create an ssh tunnel or modify the security group settings for that node to allow access to port 10005 from the machine where your jdbc client is running.
Please let us know if you need further instructions on the steps.
cc: @Dominika Bialek
Thanks for your update. I believe it is not that easy to connect to spark as you said, since I tried the same for hive. The steps you mentioned may work for the normal hortonworks setup not the cloud one. Finally realised, I have to follow the documentation by using SSL port 8443 and it perfectly worked for me to connect hive.
I guess I have to connect spark the same way I did for hive . On ambari the spark port is 10016 ( my bad I mentioned 10005) and I opened the same port on security group as per your steps. But still unable to connect. The master node is already added as a component when I checked.
Please help me.