Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Zookeeper connection with Hbase

avatar
Explorer

Hi, i'm importing data from oracle db to hbase with sqoop.
I used  sqoop line command, and all it's done.

 

sqoop import --connect jdbc:oracle:thin:@XXX:port/XXX --username XXX --password XXX --table XXX -m 1 --incremental lastmodified --last-value '2014-06-23' --check-column XXX --append --hbase-table XXX --column-family info --hbase-row-key XXX  --hbase-bulkload

 I tryed with oozie, and i have got this error in loop:

INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181.
Will not attempt to authenticate using SASL (unknown error)
WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error,
closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused

 My cluster configuration is with 5 host:
master,slave1,slave2,slave3,

slave4

sqoop 1 client:
slave1,slave2,slave3,slave4

zookeeper server:
master,slave1,slave2

hbase server:
hbaserestserver -> master
hbasethriftserver -> master
master -> master
regionserver -> slave1,slave2,slave3,slave4

I have some doubt on my configuration:
if my configuration is all ok, my question is:
zookeeper server with localhost/127.0.0.1:2181 is correct to start?
because i've got only 3 zookeeper server and i think this is my problem.
 
And there is some SQOOP_CLASSPATH or HBASE_CLASSPATH to set? oozie don't initialize everything?

Thanks
1 ACCEPTED SOLUTION

avatar
Mentor
Since Oozie lacks knowledge of where your HBase configs lie, you will need to pass the client hbase-site.xml file (placed somewhere on HDFS, by copying from /etc/hbase/conf/hbase-site.xml on any HBase gateway node) via the <job-xml>…</job-xml> option.

Alternatively, try the below command instead (will not be sufficient for secured clusters, which need further properties), replacing zk-host1,zk-host2,zk-host3 with your actual 3 hosts appropriately:

sqoop import -Dhbase.zookeeper.quorum=zk-host1,zk-host2,zk-host3 --connect jdbc:oracle:thin:@XXX:port/XXX --username XXX --password XXX --table XXX -m 1 --incremental lastmodified --last-value '2014-06-23' --check-column XXX --append --hbase-table XXX --column-family info --hbase-row-key XXX --hbase-bulkload

View solution in original post

1 REPLY 1

avatar
Mentor
Since Oozie lacks knowledge of where your HBase configs lie, you will need to pass the client hbase-site.xml file (placed somewhere on HDFS, by copying from /etc/hbase/conf/hbase-site.xml on any HBase gateway node) via the <job-xml>…</job-xml> option.

Alternatively, try the below command instead (will not be sufficient for secured clusters, which need further properties), replacing zk-host1,zk-host2,zk-host3 with your actual 3 hosts appropriately:

sqoop import -Dhbase.zookeeper.quorum=zk-host1,zk-host2,zk-host3 --connect jdbc:oracle:thin:@XXX:port/XXX --username XXX --password XXX --table XXX -m 1 --incremental lastmodified --last-value '2014-06-23' --check-column XXX --append --hbase-table XXX --column-family info --hbase-row-key XXX --hbase-bulkload