sqoop 2 - connect to netezza

[#1]

I am using cloudera 5.7.1 default installation for a 4 node cluster in linux 

I am trying to connect to netezza database but I see that I have only sqoop2 and no sqoop.

There seems to be limited documentation for sqoop2.

$ sqoop2 import --connect jdbc:netezza://<host-name>:5480/<database-name> --table test --username admin --password xxxx
Sqoop home directory: /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/sqoop2
Exception in thread "main" java.io.FileNotFoundException: /home/<username>/import (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileReader.<init>(FileReader.java:72)
at org.apache.sqoop.shell.SqoopShell.interpretFileContent(SqoopShell.java:150)
at org.apache.sqoop.shell.SqoopShell.main(SqoopShell.java:138)

[#2]

You do have sqoop installed. Just use "sqoop" instead of "sqoop2". Sqoop2 is an entirely different approach and you're instead looking to use Sqoop1 via the "sqoop" command:

sqoop import --connect jdbc:netezza://:5480/ --table test --username admin --password xxxx

If you're facing an error with the regular Sqoop command, please post back details on it including the full error and logs observed.

P.s. Sqoop2, if you are still interested in that, is documented at http://archive.cloudera.com/cdh5/cdh/5/sqoop2/Sqoop5MinutesDemo.html

[#3]

I am getting this error .

$sqoop
ERROR: Detected incomplete Hadoop installation in /usr/lib/hadoop. This is likely because you have not removed packages in a parcel environment, or you have a leftover /usr/lib/hadoop directory. If you are using parcels, please remove the /usr/lib/hadoop directory.

[#4]

When I check the directory its there. with another empty directory.

$ cd /usr/lib/hadoop
$ ls -ltr
total 4
drwxr-xr-x 2 root root 4096 Jun 7 19:25 libexec
$ cd libexec/
$ ls -ltr
total 0

I am not sure if its ok to remove this directory.

[#5]

I deleted the directory and copied the netezza jdbc driver .- nzjdbc3.jar to /var/lib/sqoop

$ sqoop import --connect jdbc:netezza://host:5480/database --table test --username xxx --password xxxx -m 1 --target-dir <hdfs dir name>
Warning: /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/08/03 22:30:25 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.7.1
16/08/03 22:30:25 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/08/03 22:30:25 INFO manager.SqlManager: Using default fetchSize of 1000
16/08/03 22:30:25 INFO tool.CodeGenTool: Beginning code generation
16/08/03 22:30:26 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "test" AS t WHERE 1=0
16/08/03 22:30:26 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "test" AS t WHERE 1=0
16/08/03 22:30:26 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
Note: /tmp/sqoop-xxxx/compile/ccdd1201f74910d4a09ab951551e89b7/test.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
16/08/03 22:30:27 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-xxxx/compile/ccdd1201f74910d4a09ab951551e89b7/test.jar
16/08/03 22:30:27 WARN manager.NetezzaManager: It looks like you are importing from Netezza.
16/08/03 22:30:27 WARN manager.NetezzaManager: This transfer can be faster! Use the --direct
16/08/03 22:30:27 WARN manager.NetezzaManager: option to exercise a Netezza-specific fast path.
16/08/03 22:30:29 INFO mapreduce.ImportJobBase: Beginning import of test
Exception in thread "main" java.lang.NoClassDefFoundError: org/json/JSONObject
at org.apache.sqoop.util.SqoopJsonUtil.getJsonStringforMap(SqoopJsonUtil.java:42)
at org.apache.sqoop.SqoopOptions.writeProperties(SqoopOptions.java:742)
at org.apache.sqoop.mapreduce.JobBase.putSqoopOptionsToConfiguration(JobBase.java:369)
at org.apache.sqoop.mapreduce.JobBase.createJob(JobBase.java:355)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:249)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
at org.apache.sqoop.manager.NetezzaManager.importTable(NetezzaManager.java:117)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: java.lang.ClassNotFoundException: org.json.JSONObject
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 15 more

[#6]

fixed the above error.

Now I am getting this error below.

$ sqoop import --connect jdbc:netezza:xxx--table test --username admin --password xxxx -m 1
Warning: /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/08/04 00:07:34 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.7.1
16/08/04 00:07:34 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/08/04 00:07:34 INFO manager.SqlManager: Using default fetchSize of 1000
16/08/04 00:07:34 INFO tool.CodeGenTool: Beginning code generation
16/08/04 00:07:35 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "test" AS t WHERE 1=0
16/08/04 00:07:35 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "test" AS t WHERE 1=0
16/08/04 00:07:35 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
Note: /tmp/sqoop-xxxx/compile/8b207183e16b331a49cdb5264b594657/test.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
16/08/04 00:07:37 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-xxx/compile/8b207183e16b331a49cdb5264b594657/test.jar
16/08/04 00:07:37 WARN manager.NetezzaManager: It looks like you are importing from Netezza.
16/08/04 00:07:37 WARN manager.NetezzaManager: This transfer can be faster! Use the --direct
16/08/04 00:07:37 WARN manager.NetezzaManager: option to exercise a Netezza-specific fast path.
16/08/04 00:07:39 INFO mapreduce.ImportJobBase: Beginning import of test
16/08/04 00:07:39 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
16/08/04 00:07:39 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM "test" AS t WHERE 1=0
16/08/04 00:07:41 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
16/08/04 00:07:41 INFO client.RMProxy: Connecting to ResourceManager at xxxx
16/08/04 00:07:44 INFO db.DBInputFormat: Using read commited transaction isolation
16/08/04 00:07:44 INFO mapreduce.JobSubmitter: number of splits:1
16/08/04 00:07:44 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1467176512921_0007
16/08/04 00:07:45 INFO impl.YarnClientImpl: Submitted application application_1467176512921_0007
16/08/04 00:07:45 INFO mapreduce.Job: The url to track the job: xxxx
16/08/04 00:07:45 INFO mapreduce.Job: Running job: job_1467176512921_0007
16/08/04 00:07:52 INFO mapreduce.Job: Job job_1467176512921_0007 running in uber mode : false
16/08/04 00:07:52 INFO mapreduce.Job: map 0% reduce 0%
16/08/04 00:07:59 INFO mapreduce.Job: Task Id : attempt_1467176512921_0007_m_000000_0, Status : FAILED
Error: test : Unsupported major.minor version 52.0
16/08/04 00:08:05 INFO mapreduce.Job: Task Id : attempt_1467176512921_0007_m_000000_1, Status : FAILED
Error: test : Unsupported major.minor version 52.0
16/08/04 00:08:09 INFO mapreduce.Job: Task Id : attempt_1467176512921_0007_m_000000_2, Status : FAILED
Error: test : Unsupported major.minor version 52.0
16/08/04 00:08:17 INFO mapreduce.Job: map 100% reduce 0%
16/08/04 00:08:17 INFO mapreduce.Job: Job job_1467176512921_0007 failed with state FAILED due to: Task failed task_1467176512921_0007_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

16/08/04 00:08:17 INFO mapreduce.Job: Counters: 11
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=16982
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=16982
Total vcore-seconds taken by all map tasks=16982
Total megabyte-seconds taken by all map tasks=17389568
Map-Reduce Framework
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
16/08/04 00:08:17 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
16/08/04 00:08:17 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 36.1369 seconds (0 bytes/sec)
16/08/04 00:08:17 INFO mapreduce.ImportJobBase: Retrieved 0 records.
16/08/04 00:08:17 ERROR tool.ImportTool: Error during import: Import job failed!
$