question Re: The node /hbase is not in ZooKeeper. It should have been written by the master. in Archives of Support Questions (Read Only)

The node /hbase is not in ZooKeeper. It should have been written by the master.

zachkirsch — Fri, 24 Jun 2016 03:32:51 GMT

I am trying to connect to HBase from Spark via Phoenix (using the official plugin described here).

Here is my scala Spark code:

package com.zachk

import org.apache.spark.SparkContext
import org.apache.spark.sql.SQLContext
import org.apache.phoenix.spark._
import org.apache.spark.rdd.RDD

object App {
  def main(args : Array[String]) {
    val sc = new SparkContext()
    val rdd: RDD[Map[String, AnyRef]] = sc.phoenixTableAsRDD(
      //"TABLE1", Seq("ID", "COL1"), zkUrl = Some("zk1-titanu:2181/hbase-unsecure")
      //"TABLE1", Seq("ID", "COL1"), zkUrl = Some("zk1-titanu:2181/hbase-secure")
      "TABLE1", Seq("ID", "COL1"), zkUrl = Some("zk1-titanu:2181")
    )
    println(rdd.count())
}

I am compiling with Maven.

I am running with this spark-submit command:

/usr/bin/spark-submit --class com.zachk.App --master local[*] target/phoenix-hbase-test-0.0.1-jar-with-dependencies.jar

When I run, I get this error continually: ERROR ConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.

As shown in the comments above, I have already tried the suggestion in this similarly-named post. I would appreciate any help!

Re: The node /hbase is not in ZooKeeper. It should have been written by the master.

elserj — Fri, 24 Jun 2016 03:32:52 GMT

The zkUrl you provided the first time should be correct:

"zk1-titanu:2181/hbase-unsecure"

By default, HBase on HDP will use /hbase-unsecure. If you enable Kerberos authentication, it will use /hbase-secure instead.

Re: The node /hbase is not in ZooKeeper. It should have been written by the master.

ssoldatov — Fri, 24 Jun 2016 03:36:23 GMT

The problem is in missing colons in the URLs you tried. It's supposed to be "zk1-titanu:2181:/hbase-unsecure"

Re: The node /hbase is not in ZooKeeper. It should have been written by the master.

tyu — Fri, 24 Jun 2016 03:39:13 GMT

The functions `phoenixTableAsDataFrame`, `phoenixTableAsRDD` and `saveToPhoenix` all support optionally specifying a `conf` Hadoop configuration parameter with custom Phoenix client settings, as well as an optional `zkUrl` parameter for the Phoenix connection URL.

val configuration = new Configuration()

// set zookeeper.znode.parent and hbase.zookeeper.quorum in the conf

"TABLE1", Seq("ID", "COL1"), conf = configuration

Re: The node /hbase is not in ZooKeeper. It should have been written by the master.

rgelhausen — Fri, 24 Jun 2016 03:42:56 GMT

You're missing a ':' after 2181