Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Zookeeper connection with Hbase

avatar
Explorer

Hi, i'm importing data from oracle db to hbase with sqoop.
I used  sqoop line command, and all it's done.

 

sqoop import --connect jdbc:oracle:thin:@XXX:port/XXX --username XXX --password XXX --table XXX -m 1 --incremental lastmodified --last-value '2014-06-23' --check-column XXX --append --hbase-table XXX --column-family info --hbase-row-key XXX  --hbase-bulkload

 I tryed with oozie, and i have got this error in loop:

INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181.
Will not attempt to authenticate using SASL (unknown error)
WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error,
closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused

 My cluster configuration is with 5 host:
master,slave1,slave2,slave3,

slave4

sqoop 1 client:
slave1,slave2,slave3,slave4

zookeeper server:
master,slave1,slave2

hbase server:
hbaserestserver -> master
hbasethriftserver -> master
master -> master
regionserver -> slave1,slave2,slave3,slave4

I have some doubt on my configuration:
if my configuration is all ok, my question is:
zookeeper server with localhost/127.0.0.1:2181 is correct to start?
because i've got only 3 zookeeper server and i think this is my problem.
 
And there is some SQOOP_CLASSPATH or HBASE_CLASSPATH to set? oozie don't initialize everything?

Thanks
1 ACCEPTED SOLUTION

avatar
Mentor
Since Oozie lacks knowledge of where your HBase configs lie, you will need to pass the client hbase-site.xml file (placed somewhere on HDFS, by copying from /etc/hbase/conf/hbase-site.xml on any HBase gateway node) via the <job-xml>…</job-xml> option.

Alternatively, try the below command instead (will not be sufficient for secured clusters, which need further properties), replacing zk-host1,zk-host2,zk-host3 with your actual 3 hosts appropriately:

sqoop import -Dhbase.zookeeper.quorum=zk-host1,zk-host2,zk-host3 --connect jdbc:oracle:thin:@XXX:port/XXX --username XXX --password XXX --table XXX -m 1 --incremental lastmodified --last-value '2014-06-23' --check-column XXX --append --hbase-table XXX --column-family info --hbase-row-key XXX --hbase-bulkload

View solution in original post

1 REPLY 1

avatar
Mentor
Since Oozie lacks knowledge of where your HBase configs lie, you will need to pass the client hbase-site.xml file (placed somewhere on HDFS, by copying from /etc/hbase/conf/hbase-site.xml on any HBase gateway node) via the <job-xml>…</job-xml> option.

Alternatively, try the below command instead (will not be sufficient for secured clusters, which need further properties), replacing zk-host1,zk-host2,zk-host3 with your actual 3 hosts appropriately:

sqoop import -Dhbase.zookeeper.quorum=zk-host1,zk-host2,zk-host3 --connect jdbc:oracle:thin:@XXX:port/XXX --username XXX --password XXX --table XXX -m 1 --incremental lastmodified --last-value '2014-06-23' --check-column XXX --append --hbase-table XXX --column-family info --hbase-row-key XXX --hbase-bulkload