Support Questions

Find answers, ask questions, and share your expertise

Sqoop import table from mysql to Hive failled:java.util.LinkedList cannot be cast to org.apache.hive.hcatalog.mapreduce.InputJobInfo

avatar
New Contributor

I am trying to import a table from mysql to hive and i getting this errors(all 23 errors almost the same).

 

023-01-09 18:00:15,370 INFO mapreduce.Job: Task Id : attempt_1673252459679_0002_m_000001_0, Status : FAILED Error: java.lang.ClassCastException: java.util.LinkedList cannot be cast to org.apache.hive.hcatalog.mapreduce.InputJobInfo at org.apache.sqoop.mapreduce.hcat.SqoopHCatImportHelper.<init>(SqoopHCatImportHelper.java:88) at org.apache.sqoop.mapreduce.hcat.SqoopHCatImportMapper.setup(SqoopHCatImportMapper.java:45) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:800) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:348) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:178) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)

 

 

the command is:

sqoop import -connect jdbc:mysql://master:3306/*** --username **** --password **** --query "select id,referrer,from_url,landing_page_url,url_title,platform_description,other_params,history from web_chat_text_ems_2019_07 where 1=1 and $CONDITIONS" --hcatalog-database itcast_ods --hcatalog-table web_chat_text_ems_ods -m 10 --split-by id

 

the versions: sqoop 1.4.7, hadoop 3.3.1 hive4.0.0-alpha-2,mysql8.0.

And use the command

sqoop list-databases -connect jdbc:mysql://master:3306/ --username **** --password ****

is ok.

 

why have the errors and how to solve it.

thanks 

1 REPLY 1

avatar
Master Collaborator

The error message you're encountering, "java.util.LinkedList cannot be cast to org.apache.hive.hcatalog.mapreduce.InputJobInfo," typically occurs when there's a mismatch between the data types or structures during a Sqoop import from MySQL to Hive. Here are some steps to troubleshoot and potentially resolve the issue:

  1. Check Hive and HCatalog Compatibility: Ensure that the versions of Hive and HCatalog you are using are compatible with your version of Sqoop. If there's a version mismatch, consider updating or downgrading one of them to ensure compatibility.

  2. Check Your SQL Query: Review the SQL query you're using with Sqoop to import data. Ensure that it's correctly configured and that the source and target tables are correctly specified.

  3. Check Data Types: Ensure that the data types of the source MySQL table match the data types of the target Hive table. Inconsistencies in data types can lead to this error.

  4. Check Field Mapping: Verify that the fields in your MySQL table match the columns in your Hive table in terms of number and order. Ensure there are no extra or missing columns.