Reply
Highlighted
Contributor
Posts: 26
Registered: ‎05-12-2015
Accepted Solution

Unable to use sqoop on cloudera CDH5

Hi team,

 

I am getting below error while trying to use sqoop syntax. Unable to use sqoop on cloudera CDH5

 

root@blrrndtipamo02:/opt/cloudera/parcels/CDH/lib/sqoop/bin# sqoop version
Error: /usr/lib/sqoop/../hadoop does not exist!
Please set $HADOOP_COMMON_HOME to the root of your Hadoop installation.

 

 

root@blrrndtipamo02:/opt/cloudera/parcels/CDH/lib/sqoop/bin# sqoop help
Error: /usr/lib/sqoop/../hadoop does not exist!
Please set $HADOOP_COMMON_HOME to the root of your Hadoop installation.

 

Please suggest

Cloudera Employee
Posts: 92
Registered: ‎08-01-2013

Re: Unable to use sqoop on cloudera CDH5

What does 'hadoop version' return on that host?

Are you deploying HDFS or YARN gateway role to that host? See http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_mc_managing_roles.ht...

Contributor
Posts: 26
Registered: ‎05-12-2015

Re: Unable to use sqoop on cloudera CDH5

CDH5 is installed

 

The above issue got fixed, now I get below issue while  trying to run sqoop syntax.

 

sqoop import --verbose --connect jdbc:oracle://hostname.net:15012/mid --username srpnbhat --password Bnmae3@ --table SRP_RUN --target-dir /user/hive/warehouse/nag

 

 

<script src="res://instant-message.dll/2438" type="text/javascript"></script>

15/05/13 11:20:16 DEBUG manager.OracleManager: Using column names query: SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 11:20:16 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 11:20:16 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:286)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:736)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:759)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:269)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)

 

Please suggest if we are missing anything here

Cloudera Employee
Posts: 92
Registered: ‎08-01-2013

Re: Unable to use sqoop on cloudera CDH5

Have you installed the Oracle JDBC driver per the following instructions?

 

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_jdbc_driver_inst...

Contributor
Posts: 26
Registered: ‎05-12-2015

Re: Unable to use sqoop on cloudera CDH5

Hi

 

As per steps on above link  I did not have dir as /usr/lib/sqoop/lib. I create dir sqoop/lib and copied the ojdbc6_g.jar file

 

Still I get same error 

 

15/05/13 13:10:51 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.3
15/05/13 13:10:51 DEBUG tool.BaseSqoopTool: Enabled debug logging.
15/05/13 13:10:51 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Loaded manager factory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Trying ManagerFactory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 13:10:51 DEBUG oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop can be called by Sqoop!
15/05/13 13:10:51 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disabled.
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 13:10:51 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:oracle:
15/05/13 13:10:51 DEBUG manager.OracleManager$ConnCache: Instantiated new connection cache.
15/05/13 13:10:51 INFO manager.SqlManager: Using default fetchSize of 1000
15/05/13 13:10:51 DEBUG sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.OracleManager@8297b3a
15/05/13 13:10:51 INFO tool.CodeGenTool: Beginning code generation
15/05/13 13:10:51 DEBUG manager.OracleManager: Using column names query: SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 13:10:51 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 13:10:51 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:286)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:736)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:759)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:269)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

Cloudera Employee
Posts: 92
Registered: ‎08-01-2013

Re: Unable to use sqoop on cloudera CDH5

Can you rename the jar file to ojdbc6.jar and give it a try?

Contributor
Posts: 26
Registered: ‎05-12-2015

Re: Unable to use sqoop on cloudera CDH5

Still same error

 

 

root@blrrndtipamo01:/usr/lib/sqoop/lib# mv ojdbc6_g.jar ojdbc6.jar
root@blrrndtipamo01:/usr/lib/sqoop/lib# ls
ojdbc6.jar
root@blrrndtipamo01:/usr/lib/sqoop/lib# sudo sqoop import --verbose --connect jdbc:oracle://hostname.net:15012/mid --username srpnbhat --password xxxxxxx--table SRP_RUN --target-dir /user/hive/warehouse/nag
Warning: /opt/cloudera/parcels/CDH-5.3.3-1.cdh5.3.3.p0.5/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
15/05/13 15:33:56 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.3
15/05/13 15:33:56 DEBUG tool.BaseSqoopTool: Enabled debug logging.
15/05/13 15:33:56 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Loaded manager factory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Trying ManagerFactory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
15/05/13 15:33:56 DEBUG oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop can be called by Sqoop!
15/05/13 15:33:56 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop is disabled.
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory
15/05/13 15:33:56 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:oracle:
15/05/13 15:33:56 DEBUG manager.OracleManager$ConnCache: Instantiated new connection cache.
15/05/13 15:33:56 INFO manager.SqlManager: Using default fetchSize of 1000
15/05/13 15:33:56 DEBUG sqoop.ConnFactory: Instantiated ConnManager org.apache.sqoop.manager.OracleManager@8297b3a
15/05/13 15:33:56 INFO tool.CodeGenTool: Beginning code generation
15/05/13 15:33:56 DEBUG manager.OracleManager: Using column names query: SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 15:33:56 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM SRP_RUN t WHERE 1=0
15/05/13 15:33:56 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
java.lang.RuntimeException: Could not load db driver class: oracle.jdbc.OracleDriver
at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:286)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:736)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:759)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:269)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:240)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:226)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

Cloudera Employee
Posts: 92
Registered: ‎08-01-2013

Re: Unable to use sqoop on cloudera CDH5

Is the permission configured correctly?

 

chown sqoop:sqoop /var/lib/sqoop 
chmod 755 /var/lib/sqoop
Contributor
Posts: 26
Registered: ‎05-12-2015

Re: Unable to use sqoop on cloudera CDH5

Yes, prerequities already taken care.

 

premission check

drwxr-xr-x 2 sqoop sqoop 4.0K May 13 13:42 sqoop

Contributor
Posts: 26
Registered: ‎05-12-2015

Re: Unable to use sqoop on cloudera CDH5

Please let me know if anything to be taken care in configuration files...such as hdfs setup or Hive.....namenode format

 

 

CDH5 on Debain linux OS

 

Expecting the sqoop to import oracle data into HDFS. Please suggest

Announcements