Hi Eric,   Thanks for your response.   when i ran the below command: sudo -u yarn yarn logs -applicationId {application_1578980062850_0002}   it is also throwing error: Invalid ApplicationId specified   please suggest me what i need to do. ... View more

Hi,   yes,configured Sqoop gateway on both the hosts.   please tell me how to run the Sqoop saved jobs in master node itself.   Thanks, Akhila. ... View more

Yes ,the job is created in Slave node  not in Master node.   There is no difference b/w the hosts.   Could you please tell me where the problem was?   Thanks, Akhila. ... View more

When i tried to run the same job from other node Job is created    What is the problem here with the master node?Job is executed from the slave node..   could you please explain me in detail...     Thanks, Akhila. ... View more

Hi,   We didn't install it manually . I think in the CDH cluster installation itself sqoop metastore is also included.   No we didn't change any sqoop configurations ..   When i ran the command sudo -u sqoop sqoop-metastore   output: 19/08/26 11:55:48 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.16.1 [Server@486f90fe]: [Thread[main,5,main]]: checkRunning(false) entered [Server@486f90fe]: [Thread[main,5,main]]: checkRunning(false) exited [Server@486f90fe]: [Thread[main,5,main]]: setDatabasePath(0,file:/usr/lib/sqoop/metastore/) [Server@486f90fe]: [Thread[main,5,main]]: checkRunning(false) entered [Server@486f90fe]: [Thread[main,5,main]]: checkRunning(false) exited [Server@486f90fe]: [Thread[main,5,main]]: setDatabaseName(0,sqoop) [Server@486f90fe]: [Thread[main,5,main]]: putPropertiesFromString(): [hsqldb.write_delay=false] [Server@486f90fe]: [Thread[main,5,main]]: checkRunning(false) entered [Server@486f90fe]: [Thread[main,5,main]]: checkRunning(false) exited [Server@486f90fe]: Initiating startup sequence... [Server@486f90fe]: Server socket opened successfully in 6 ms. [Server@486f90fe]: [Thread[HSQLDB Server @486f90fe,5,main]]: Database [index=0db=file:/usr/lib/sqoop/metastore/, alias=sqoop] did not open: org.hsqldb.HsqlException: File input/output error /usr/lib/sqoop/metastore/.properties java.io.FileNotFoundException: /usr/lib/sqoop/metastore/.properties.new (No such file or directory) [Server@486f90fe]: [Thread[HSQLDB Server @486f90fe,5,main]]: Shutting down because there are no open databases [Server@486f90fe]: Initiating shutdown sequence... [Server@486f90fe]: Shutdown sequence completed in 2 ms. [Server@486f90fe]: 2019-08-26 11:55:48.641 SHUTDOWN : System.exit() was not called 19/08/26 11:55:48 INFO hsqldb.HsqldbMetaStore: Server started on port 16000 with protocol HSQL 19/08/26 11:55:48 INFO hsqldb.HsqldbMetaStore: Got shutdown notification 19/08/26 11:55:48 INFO tool.MetastoreTool: Server thread has quit.   Thanks, Akhila. ... View more

Hi,   We Installed CDH cluster in our environment not the individual components.    We are able to run the normal sqoop commands and got the output also.. the problem is with sqoop saved jobs only.   when i ran the sqoop job --list we got the same error:   19/08/26 11:17:18 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.16.1 19/08/26 11:17:18 ERROR tool.JobTool: I/O error performing job operation: java.io.IOException: Exception creating SQL connection at org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.init(HsqldbJobStorage.java:217) at org.apache.sqoop.metastore.hsqldb.AutoHsqldbStorage.open(AutoHsqldbStorage.java:112) at org.apache.sqoop.tool.JobTool.run(JobTool.java:290) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) Caused by: java.sql.SQLException: socket creation error at org.hsqldb.jdbc.Util.sqlException(Unknown Source) at org.hsqldb.jdbc.jdbcConnection.<init>(Unknown Source) at org.hsqldb.jdbcDriver.getConnection(Unknown Source) at org.hsqldb.jdbcDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.init(HsqldbJobStorage.java:177) ... 8 more   we have 400 GB disk space available in that host.   Thanks, Akhila. ... View more

Hi,   Sqoop is not there in var/lib/sqoop  Sqoop is installed in /opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/sqoop in my environment.   so i ran the command: ls -ltr  /opt/cloudera/parcels/CDH-5.16.1-1.cdh5.16.1.p0.3/lib/sqoop   output:   total 600 -rw-r--r-- 1 root root 505 Nov 22 2018 NOTICE.txt -rw-r--r-- 1 root root 15419 Nov 22 2018 LICENSE.txt drwxr-xr-x 2 root root 4096 Nov 22 2018 cloudera lrwxrwxrwx 1 root root 30 Nov 22 2018 sqoop-test.jar -> sqoop-test-1.4.6-cdh5.16.1.jar lrwxrwxrwx 1 root root 25 Nov 22 2018 sqoop.jar -> sqoop-1.4.6-cdh5.16.1.jar lrwxrwxrwx 1 root root 15 Nov 22 2018 conf -> /etc/sqoop/conf lrwxrwxrwx 1 root root 41 Nov 22 2018 sqoop-test-1.4.6-cdh5.16.1.jar -> ../../jars/sqoop-test-1.4.6-cdh5.16.1.jar lrwxrwxrwx 1 root root 36 Nov 22 2018 sqoop-1.4.6-cdh5.16.1.jar -> ../../jars/sqoop-1.4.6-cdh5.16.1.jar drwxr-xr-x 2 root root 4096 Nov 22 2018 lib -rw-r--r-- 1 root root 579785 Jul 19 16:48 postgresql-9.2-1002.jdbc4.jar drwxr-xr-x 3 root root 4096 Aug 23 17:45 bin   ls -ltra /home/ubuntu   output:   total 931732 -rw-r--r-- 1 ubuntu ubuntu 675 Nov 22 2018 .profile -rw-r--r-- 1 ubuntu ubuntu 3637 Nov 22 2018 .bashrc -rw-r--r-- 1 ubuntu ubuntu 220 Nov 22 2018 .bash_logout drwx------ 2 ubuntu ubuntu 4096 Nov 22 2018 .cache -rw------- 1 ubuntu ubuntu 28 Nov 22 2018 .bash_history drwxr-xr-x 3 root root 4096 Dec 4 2018 elasticsearch-hadoop-6.3.2 drwxr-xr-x 5 root root 4096 Feb 18 2019 venv -rw-r--r-- 1 root root 470003663 May 10 14:35 signalstrength_20190510.backup drwxr-xr-x 3 root root 4096 Aug 7 14:37 .. -rw-r--r-- 1 root root 470994315 Aug 20 15:45 inet.sql -rw-r--r-- 1 root root 13040148 Aug 21 17:42 npa.sql drwxr-xr-x 2 root root 4096 Aug 23 12:31 python -rw-r--r-- 1 root root 323 Aug 23 17:46 script.sh drwxr-xr-x 6 ubuntu ubuntu 4096 Aug 23 17:46 .     Regards, Akhila.     ... View more

Hi,   The output of ls -ltr is:   total 600 -rw-r--r-- 1 root root 505 Nov 22 2018 NOTICE.txt -rw-r--r-- 1 root root 15419 Nov 22 2018 LICENSE.txt drwxr-xr-x 2 root root 4096 Nov 22 2018 cloudera lrwxrwxrwx 1 root root 30 Nov 22 2018 sqoop-test.jar -> sqoop-test-1.4.6-cdh5.16.1.jar lrwxrwxrwx 1 root root 25 Nov 22 2018 sqoop.jar -> sqoop-1.4.6-cdh5.16.1.jar lrwxrwxrwx 1 root root 15 Nov 22 2018 conf -> /etc/sqoop/conf lrwxrwxrwx 1 root root 41 Nov 22 2018 sqoop-test-1.4.6-cdh5.16.1.jar -> ../../jars/sqoop-test-1.4.6-cdh5.16.1.jar lrwxrwxrwx 1 root root 36 Nov 22 2018 sqoop-1.4.6-cdh5.16.1.jar -> ../../jars/sqoop-1.4.6-cdh5.16.1.jar drwxr-xr-x 2 root root 4096 Nov 22 2018 lib -rw-r--r-- 1 root root 579785 Jul 19 16:48 postgresql-9.2-1002.jdbc4.jar drwxr-xr-x 3 root root 4096 Aug 23 17:45 bin     could you please check and resolve the issue.....     Thanks, Akhila.       ... View more

Hi Eric,   When we tried to create sqoop saved jobs through this command:   sqoop job --create  myjob -- import --connect jdbc:postgresql://ipaddress:5432/inet --username postgres --P --table company --fields-terminated-by '|' --lines-terminated-by '\n' --hive-import --hive-table sampletest    It throws the following error:   19/08/16 15:52:03 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.16.1 19/08/16 15:52:04 ERROR tool.JobTool: I/O error performing job operation: java.io.IOException: Exception creating SQL connection at org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.init(HsqldbJobStorage.java:217) at org.apache.sqoop.metastore.hsqldb.AutoHsqldbStorage.open(AutoHsqldbStorage.java:112) at org.apache.sqoop.tool.JobTool.run(JobTool.java:290) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) Caused by: java.sql.SQLException: socket creation error at org.hsqldb.jdbc.Util.sqlException(Unknown Source) at org.hsqldb.jdbc.jdbcConnection.<init>(Unknown Source) at org.hsqldb.jdbcDriver.getConnection(Unknown Source) at org.hsqldb.jdbcDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:215) at org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.init(HsqldbJobStorage.java:177) ... 8 more    Could you please help me in resolving the issue....   ... View more