Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop import 'possible' tables

Sqoop import 'possible' tables

Expert Contributor

HDP-2.4.2.0-258 installed using Ambari 2.2.2.0.

I have more than 15 sql server db schema to be imported, each containing more 60 tables.

I checked the existing threads like this and this but I am a bit unclear about how to proceed, maybe there are better ways to do the import which I am unaware of. If I launch 4-5 Sqoop imports simult., this will trigger concurrent MR jobs, is there a way to let Sqoop/Hadoop handle the asynchronous execution of the Sqoop scripts ?

If I consider the Sqoop documentation about importing all tables , the following scenarios exist :

  • Not ALL tables have PKs
  • Some tables have a varchar PK
  • Most of the tables have composite PK

What is the fastest way to import these large schema(s) ?

2 REPLIES 2

Re: Sqoop import 'possible' tables

@Kaliyug Antagonist

You can try using sqoop import-all-tables with --autoreset-to-one-mapper and check if it helps.

https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_literal_sqoop_import_all_tables_literal

Highlighted

Re: Sqoop import 'possible' tables

Hi, what if you want to import from multiple tables using joins? so basically if i have three tables from different schemas and i want to query all three and import into HIVE using sqoop. Is this possible?

Don't have an account?
Coming from Hortonworks? Activate your account here