Member since
01-24-2017
2
Posts
1
Kudos Received
0
Solutions
03-02-2017
09:11 AM
1 Kudo
I have to use sqoop to import all tables from a mysql database to hdfs and to external tables in hive (no filters, with the same structure) In import I want to bring:
New data for existing tables Updated data for existing tables (using only the id column) New tables created in mysql (y to create external table in hive) Then create a sqoop job to do it all automatically. (I have a mysql database with approximately 60 tables, and with each new client going into production, a new table is created. So I need sqoop to work as automatically as possible) The first command executed to import all the tables was: sqoop import-all-tables --connect jdbc:mysql://IP/db_name --username user
--password pass --warehouse-dir /user/hdfs/db_name
-m 1 Here Scoop and support for external Hive tables says that support was added for the creation of external tables in hive , but I did not find documentation or examples on the mentioned commands
What are the best practices to work with in sqoop where it looks at all the updates from a mysql database and passes to hdfs and hive ? Any ideas would be good. Thanks in advance.
... View more
Labels: