Unable to use sqoop on cloudera CDH5

[#1]

Hi team,

I am getting below error while trying to use sqoop syntax. Unable to use sqoop on cloudera CDH5

root@blrrndtipamo02:/opt/cloudera/parcels/CDH/lib/sqoop/bin# sqoop version
Error: /usr/lib/sqoop/../hadoop does not exist!
Please set $HADOOP_COMMON_HOME to the root of your Hadoop installation.

root@blrrndtipamo02:/opt/cloudera/parcels/CDH/lib/sqoop/bin# sqoop help
Error: /usr/lib/sqoop/../hadoop does not exist!
Please set $HADOOP_COMMON_HOME to the root of your Hadoop installation.

Please suggest

[#2]

What does 'hadoop version' return on that host?

Are you deploying HDFS or YARN gateway role to that host? See http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_mc_managing_roles.ht...

[#3]

CDH5 is installed

The above issue got fixed, now I get below issue while  trying to run sqoop syntax.

sqoop import --verbose --connect jdbc:oracle://hostname.net:15012/mid --username srpnbhat --password Bnmae3@ --table SRP_RUN --target-dir /user/hive/warehouse/nag

<script src="res://instant-message.dll/2438" type="text/javascript"></script>

15/05/13 11:20:16 DEBUG manager.OracleManager: Using column names query: SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 11:20:16 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 11:20:16 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:286)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:736)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:759)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:269)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)

Please suggest if we are missing anything here

[#4]

Have you installed the Oracle JDBC driver per the following instructions?

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_jdbc_driver_inst...

[#5]

Hi

As per steps on above link  I did not have dir as /usr/lib/sqoop/lib. I create dir sqoop/lib and copied the ojdbc6_g.jar file

Still I get same error 

15/05/13 13:10:51 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.3
15/05/13 13:10:51 DEBUG tool.BaseSqoopTool: Enabled debug logging.
15/05/13 13:10:51 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Loaded manager factory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Trying ManagerFactory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 13:10:51 DEBUG oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop can be called by Sqoop!
15/05/13 13:10:51 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disabled.
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 13:10:51 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:oracle:
15/05/13 13:10:51 DEBUG manager.OracleManager$ConnCache: Instantiated new connection cache.
15/05/13 13:10:51 INFO manager.SqlManager: Using default fetchSize of 1000
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.OracleManager@8297b3a
15/05/13 13:10:51 INFO tool.CodeGenTool: Beginning code generation
15/05/13 13:10:51 DEBUG manager.OracleManager: Using column names query: SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 13:10:51 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 13:10:51 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:286)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:736)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:759)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:269)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

[#6]

Can you rename the jar file to ojdbc6.jar and give it a try?

[#7]

Still same error

root@blrrndtipamo01:/usr/lib/sqoop/lib# mv ojdbc6_g.jar ojdbc6.jar
root@blrrndtipamo01:/usr/lib/sqoop/lib# ls
ojdbc6.jar
root@blrrndtipamo01:/usr/lib/sqoop/lib# sudo sqoop import --verbose --connect jdbc:oracle://hostname.net:15012/mid --username srpnbhat --password xxxxxxx--table SRP_RUN --target-dir /user/hive/warehouse/nag
Warning: /opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
15/05/13 15:33:56 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.3
15/05/13 15:33:56 DEBUG tool.BaseSqoopTool: Enabled debug logging.
15/05/13 15:33:56 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Loaded manager factory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Trying ManagerFactory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 15:33:56 DEBUG oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop can be called by Sqoop!
15/05/13 15:33:56 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disabled.
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 15:33:56 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:oracle:
15/05/13 15:33:56 DEBUG manager.OracleManager$ConnCache: Instantiated new connection cache.
15/05/13 15:33:56 INFO manager.SqlManager: Using default fetchSize of 1000
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.OracleManager@8297b3a
15/05/13 15:33:56 INFO tool.CodeGenTool: Beginning code generation
15/05/13 15:33:56 DEBUG manager.OracleManager: Using column names query: SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 15:33:56 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 15:33:56 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:286)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:736)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:759)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:269)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

[#8]

Is the permission configured correctly?

chown sqoop:sqoop /var/lib/sqoop 
chmod 755 /var/lib/sqoop

[#9]

Yes, prerequities already taken care.

premission check

drwxr-xr-x 2 sqoop sqoop 4.0K May 13 13:42 sqoop

[#10]

Please let me know if anything to be taken care in configuration files...such as hdfs setup or Hive.....namenode format

CDH5 on Debain linux OS

Expecting the sqoop to import oracle data into HDFS. Please suggest