Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Zookeeper error while accessing phoenix using pyspark

Zookeeper error while accessing phoenix using pyspark

New Contributor

I'm trying to read data from phoenix using pyspark. I'm getting below mentioned error:

19/06/11 17:20:12 WARN ConnectionImplementation: Retrieve cluster id failed java.util.concurrent.ExecutionException: org.apache.phoenix.shaded.org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /hbase/hbaseid     at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)     at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)     at org.apache.hadoop.hbase.client.ConnectionImplementation.retrieveClusterId(ConnectionImplementation.java:527)     at org.apache.hadoop.hbase.client.ConnectionImplementation.<init>(ConnectionImplementation.java:287)


I'm loading the pyspark as follows:

pyspark --name elasticsearch_sink --jars jars/phoenix-5.0.0.3.0.1.0-187-client.jar,jars/spark-sql-kafka-0-10_2.11-2.3.1.jar,jars/spark-streaming-kafka-0-10-assembly_2.11-2.3.1.jar,jars/elasticsearch-hadoop-6.8.0.jar --files "/home/user/spark/artifacts/jaas.conf,./user.keytab" --driver-java-options '-Djava.security.auth.login.config=/home/user/spark/artifacts/jaas.conf' --conf "spark.executor.extraJavaOptions=-Djava.security.auth.login.config=jaas.conf" --master yarn

I'm running below mentioned code:

from pyspark import SparkContext
from pyspark.sql import SQLContext
from pyspark.sql.session import SparkSession
from pyspark.sql.functions import *
from pyspark.sql.types import *
from random import randrange
import json
sc = SparkContext.getOrCreate()
spark = SparkSession.builder.appName("es_sink").getOrCreate()
sqlc = SQLContext(sc)
df = sqlc.read.format("org.apache.phoenix.spark").option("zkUrl","zookeeper:2181").option("table","proxy").load()


Don't have an account?
Coming from Hortonworks? Activate your account here