question Re: sqoop job -list command in Archives of Support Questions (Read Only)

sqoop job -list command

sim6 — Sun, 08 May 2016 21:20:51 GMT

sqoop job -list

command lists no availables jobs. I have created the jobs successfully though.

I changed my metastore to mysql and then created the jobs. Is it possible that this has something to do with metastore change? When I try to create a job with the same name that already exists, it confirms that there is a job with that name already.

1. Where are these jobs stored?

2. What are the possible reasons that it is not able to list the jobs ?

3. How do I fix this?

Re: sqoop job -list command

aervits — Sun, 08 May 2016 21:26:04 GMT

You need to make sure you're connecting to the correct metastore. Are you connecting to the shared or your personal metastore? Try to pass --meta-connect and url of metastore.

Re: sqoop job -list command

sim6 — Sun, 08 May 2016 21:27:37 GMT

Awesome. Got it 🙂

Re: sqoop job -list command

sim6 — Sun, 08 May 2016 21:28:41 GMT

Awesome. Got it 🙂 Can I make the changed metastore the default one ? so that I do not have to mention it each time? I made the changes to sqoop-site.xml though.

Re: sqoop job -list command

aervits — Sun, 08 May 2016 21:29:03 GMT

Here's user guide https://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_literal_sqoop_metastore_literal

Re: sqoop job -list command

sim6 — Sun, 08 May 2016 21:29:45 GMT

@Artem Ervits: Awesome. Got it 🙂 Can I make the changed metastore the default one ? so that I do not have to mention it each time? I made the changes to sqoop-site.xml though.

Re: sqoop job -list command

aervits — Sun, 08 May 2016 21:33:49 GMT

Yes you can, read the instructions in user guide, it needs to be a global policy.

Re: sqoop job -list command

aervits — Sun, 08 May 2016 21:36:02 GMT

Also make sure you have a way to persist metastore service as your clients will expect your metastore to be up. In Oozie job for example, you can't tell metastore is up so jobs will fail when you execute wf.

Re: sqoop job -list command

sim6 — Sun, 08 May 2016 21:37:36 GMT

@Artem Ervits : Thanks . But what does it exactly mean to persist a metastore service? What steps do I need to follow to configure that. I am using sqoop command in oozie so I shall have it that way? Do I need to configure it this way only when I am creating this as teh default metastore or whenever I have my jobs stored in a custom metastore?

Re: sqoop job -list command

aervits — Sun, 08 May 2016 21:43:52 GMT

Make sure sqoop metastore service is up. If it goes down, you need to have a way to bring it up or at least alert you. In my days, I wrote a script that would check whether mestore service was up every few minutes or so and alert if down.

Re: sqoop job -list command

sim6 — Sun, 08 May 2016 21:59:07 GMT

Thank you :). sqoop-metastore command is there but is there a command to check if the service is up and running? Something of the sort sqoop-service status? @Artem Ervits

Re: sqoop job -list command

aervits — Sun, 08 May 2016 22:24:38 GMT

No you need to implement that yourself.