Sqoop suddenly fails to connect to meta store: User not found: SA

[#1]

We are running sqoop to import data from RDBMS. We have several databases to import with thousands of tables.

We generate sqoop jobs for each table so we can use the incremental import feature of sqoop.

We are running these jobs in parallel.

Suddenly, after many successful runs, fails to connect to what I am assuming is the meta store.

Running any sqoop command causes this error. For example: sqoop job --list

We haven't explicitly set users on the metastore, and the job definition doesn't include a metastore connect command.

Any idea what could cause this and how to fix it?

Here is the error after running sqoop job --list:

Warning: /usr/hdp/2.6.2.0-205/hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /usr/hdp/2.6.2.0-205/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
17/10/26 09:27:02 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.6.2.0-205
17/10/26 09:27:03 ERROR tool.JobTool: I/O error performing job operation: java.io.IOException: Exception creating SQL connection
        at org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.init(HsqldbJobStorage.java:216)
        at org.apache.sqoop.metastore.hsqldb.AutoHsqldbStorage.open(AutoHsqldbStorage.java:112)
        at org.apache.sqoop.tool.JobTool.run(JobTool.java:289)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:243)
Caused by: java.sql.SQLException: User not found: SA
        at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
        at org.hsqldb.jdbc.jdbcConnection.<init>(Unknown Source)
        at org.hsqldb.jdbcDriver.getConnection(Unknown Source)
        at org.hsqldb.jdbcDriver.connect(Unknown Source)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.init(HsqldbJobStorage.java:176)
        ... 8 more

[#2 · Accepted solution]

I figured this out. I was trying to run concurrent sqoop jobs against the local metastore. That metastore is just a file on disk. During the concurrent runs, the file was corrupted, causing the metastore to no longer function.

We have since created a MySQL metastore, which works 90%. The last issue we are trying to resolve is it doesn't update the incremental.last.value for jobs.