Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

how can import all tables from sql server database tables to hbase using single sqoop import command? is it possible?

Contributor
 
1 ACCEPTED SOLUTION

There is no way to tell Sqoop to import all tables into HBase, becuase you have to use "--hbase-table" which is incompatible with "--import-all-tables". Note that HBase is not a general purpose data-base/storage, it's used to store a relatively small number of tables and provide real-time access to them, so it doesn't make sense to import 100s of tables into HBase. For a reasonably small number of tables you can create a script:

for t in t1 t2 t3; do
sqoop --connect jdc:mysql://... --table $t --hbase-table $t --hbase-create-table ...
done

Note that it's a good idea to pre-create HBase tables, for example to set splitting and compression etc, because Sqoop will not do that. Another approach for your project can be to import all your tables into Hive, create a few Hive tables mapped onto HBase, and populate them using your Hive imported tables.

View solution in original post

3 REPLIES 3

can you try with below command

sqoop import-all-tables --connect jdbc:mysql://mysql_server/mysql_table --table mysql_table_name --hbase-table hbase_table_name --column-family metadata --hbase-create-table --username root -P 

Explorer

Sqoop doesn’t now permit you to import, all at once, a relational table directly into an HBase table having multiple column families. To work around this limitation, you create the HBase table first and then execute Sqoop import operations to finish the task.

There is no way to tell Sqoop to import all tables into HBase, becuase you have to use "--hbase-table" which is incompatible with "--import-all-tables". Note that HBase is not a general purpose data-base/storage, it's used to store a relatively small number of tables and provide real-time access to them, so it doesn't make sense to import 100s of tables into HBase. For a reasonably small number of tables you can create a script:

for t in t1 t2 t3; do
sqoop --connect jdc:mysql://... --table $t --hbase-table $t --hbase-create-table ...
done

Note that it's a good idea to pre-create HBase tables, for example to set splitting and compression etc, because Sqoop will not do that. Another approach for your project can be to import all your tables into Hive, create a few Hive tables mapped onto HBase, and populate them using your Hive imported tables.