Created on 05-12-2015 09:12 PM - edited 09-16-2022 02:28 AM
Hi team,
I am getting below error while trying to use sqoop syntax. Unable to use sqoop on cloudera CDH5
root@blrrndtipamo02:/opt/cloudera/parcels/CDH/lib/sqoop/bin# sqoop version
Error: /usr/lib/sqoop/../hadoop does not exist!
Please set $HADOOP_COMMON_HOME to the root of your Hadoop installation.
root@blrrndtipamo02:/opt/cloudera/parcels/CDH/lib/sqoop/bin# sqoop help
Error: /usr/lib/sqoop/../hadoop does not exist!
Please set $HADOOP_COMMON_HOME to the root of your Hadoop installation.
Please suggest
Created 08-21-2015 03:04 AM
Issue fixed with below steps I performed.
Command for soft link:
ln -s /opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/lib/sqoop2/webapps/sqoop/WEB-INF/lib/derby.jar /opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/jars/derby.jar
Created 05-12-2015 10:22 PM
What does 'hadoop version' return on that host?
Are you deploying HDFS or YARN gateway role to that host? See http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_mc_managing_roles.ht...
Created 05-12-2015 11:17 PM
CDH5 is installed
The above issue got fixed, now I get below issue while trying to run sqoop syntax.
sqoop import --verbose --connect jdbc:oracle://hostname.net:15012/mid --username srpnbhat --password Bnmae3@ --table SRP_RUN --target-dir /user/hive/warehouse/nag
<script src="res://instant-message.dll/2438" type="text/javascript"></script>
15/05/13 11:20:16 DEBUG manager.OracleManager: Using column names query: SELECT t.* FROM SRP_RUN t WHERE 1=0 15/05/13 11:20:16 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM SRP_RUN t WHERE 1=0 15/05/13 11:20:16 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:286) at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:736) at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:759) at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:269) at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240) at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226) at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295) at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833) |
Please suggest if we are missing anything here
Created 05-12-2015 11:52 PM
Have you installed the Oracle JDBC driver per the following instructions?
Created 05-13-2015 12:48 AM
Hi
As per steps on above link I did not have dir as /usr/lib/sqoop/lib. I create dir sqoop/lib and copied the ojdbc6_g.jar file
Still I get same error
15/05/13 13:10:51 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.3
15/05/13 13:10:51 DEBUG tool.BaseSqoopTool: Enabled debug logging.
15/05/13 13:10:51 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Loaded manager factory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Trying ManagerFactory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 13:10:51 DEBUG oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop can be called by Sqoop!
15/05/13 13:10:51 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disabled.
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 13:10:51 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:oracle:
15/05/13 13:10:51 DEBUG manager.OracleManager$ConnCache: Instantiated new connection cache.
15/05/13 13:10:51 INFO manager.SqlManager: Using default fetchSize of 1000
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.OracleManager@8297b3a
15/05/13 13:10:51 INFO tool.CodeGenTool: Beginning code generation
15/05/13 13:10:51 DEBUG manager.OracleManager: Using column names query: SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 13:10:51 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 13:10:51 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:286)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:736)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:759)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:269)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Created 05-13-2015 01:53 AM
Can you rename the jar file to ojdbc6.jar and give it a try?
Created 05-13-2015 03:05 AM
Still same error
root@blrrndtipamo01:/usr/lib/sqoop/lib# mv ojdbc6_g.jar ojdbc6.jar
root@blrrndtipamo01:/usr/lib/sqoop/lib# ls
ojdbc6.jar
root@blrrndtipamo01:/usr/lib/sqoop/lib# sudo sqoop import --verbose --connect jdbc:oracle://hostname.net:15012/mid --username srpnbhat --password xxxxxxx--table SRP_RUN --target-dir /user/hive/warehouse/nag
Warning: /opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
15/05/13 15:33:56 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.3
15/05/13 15:33:56 DEBUG tool.BaseSqoopTool: Enabled debug logging.
15/05/13 15:33:56 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Loaded manager factory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Trying ManagerFactory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 15:33:56 DEBUG oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop can be called by Sqoop!
15/05/13 15:33:56 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disabled.
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 15:33:56 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:oracle:
15/05/13 15:33:56 DEBUG manager.OracleManager$ConnCache: Instantiated new connection cache.
15/05/13 15:33:56 INFO manager.SqlManager: Using default fetchSize of 1000
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.OracleManager@8297b3a
15/05/13 15:33:56 INFO tool.CodeGenTool: Beginning code generation
15/05/13 15:33:56 DEBUG manager.OracleManager: Using column names query: SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 15:33:56 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 15:33:56 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:286)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:736)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:759)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:269)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Created 05-13-2015 03:10 AM
Is the permission configured correctly?
chown sqoop:sqoop /var/lib/sqoop chmod 755 /var/lib/sqoop
Created 05-13-2015 04:47 AM
Yes, prerequities already taken care.
premission check
drwxr-xr-x 2 sqoop sqoop 4.0K May 13 13:42 sqoop
Created 05-13-2015 10:32 PM
Please let me know if anything to be taken care in configuration files...such as hdfs setup or Hive.....namenode format
CDH5 on Debain linux OS
Expecting the sqoop to import oracle data into HDFS. Please suggest