Created 05-08-2016 02:20 PM
sqoop job -list
command lists no availables jobs. I have created the jobs successfully though.
I changed my metastore to mysql and then created the jobs. Is it possible that this has something to do with metastore change? When I try to create a job with the same name that already exists, it confirms that there is a job with that name already.
1. Where are these jobs stored?
2. What are the possible reasons that it is not able to list the jobs ?
3. How do I fix this?
Created 05-08-2016 02:26 PM
You need to make sure you're connecting to the correct metastore. Are you connecting to the shared or your personal metastore? Try to pass --meta-connect and url of metastore.
Created 05-08-2016 02:26 PM
You need to make sure you're connecting to the correct metastore. Are you connecting to the shared or your personal metastore? Try to pass --meta-connect and url of metastore.
Created 05-08-2016 02:27 PM
Awesome. Got it 🙂
Created 05-08-2016 02:28 PM
Awesome. Got it 🙂 Can I make the changed metastore the default one ? so that I do not have to mention it each time? I made the changes to sqoop-site.xml though.
Created 05-08-2016 02:29 PM
Created 05-08-2016 02:29 PM
@Artem Ervits: Awesome. Got it 🙂 Can I make the changed metastore the default one ? so that I do not have to mention it each time? I made the changes to sqoop-site.xml though.
Created 05-08-2016 02:33 PM
Yes you can, read the instructions in user guide, it needs to be a global policy.
Created 05-08-2016 02:36 PM
Also make sure you have a way to persist metastore service as your clients will expect your metastore to be up. In Oozie job for example, you can't tell metastore is up so jobs will fail when you execute wf.
Created 05-08-2016 02:37 PM
@Artem Ervits : Thanks . But what does it exactly mean to persist a metastore service? What steps do I need to follow to configure that. I am using sqoop command in oozie so I shall have it that way? Do I need to configure it this way only when I am creating this as teh default metastore or whenever I have my jobs stored in a custom metastore?
Created 05-08-2016 02:43 PM
Make sure sqoop metastore service is up. If it goes down, you need to have a way to bring it up or at least alert you. In my days, I wrote a script that would check whether mestore service was up every few minutes or so and alert if down.