- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Multiple tables export with Sqoop in a Oozie workflow
- Labels:
-
Apache Oozie
-
Apache Sqoop
Created 06-13-2016 09:56 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
What is the best way to have an Oozie action performing a multiple tables export using Sqoop?
Context:
- I have a HDFS directory containing one file per destination table but the number of files is not constant over time
- The filename allows me to know what is the name of the target table
I'd like to have one single Oozie action calling Sqoop to export all the files in the target database.
Is it something we can do? What are the available options?
Created 06-13-2016 11:11 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ah just read that its an export. But again I don't think there is any automated way to do that.
So I suppose your only choice is to make a custom shell/ssh action that runs a script ( shell, python ... ) that lists the files from the directory and then executes sqoop jobs for them. Would be my approach.
Created 06-13-2016 11:11 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ah just read that its an export. But again I don't think there is any automated way to do that.
So I suppose your only choice is to make a custom shell/ssh action that runs a script ( shell, python ... ) that lists the files from the directory and then executes sqoop jobs for them. Would be my approach.
Created 06-13-2016 11:46 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @Benjamin Leonhardi, thanks for your answer, it confirms what I read so far. I'll go that way, thanks again.