- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
java.lang.NullPointerException at org.apache.hive.hcatalog.data.schema.HCatSchema.get
- Labels:
-
Apache HCatalog
-
Apache Hive
-
Apache Sqoop
Created 10-27-2015 08:35 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
HDP version: 2.3.0 upgraded from 2.2.4.2
A user reported his sqoop command fails if database column name and HCatalog column name are different. If he uses same column names for both side, it works. But he thinks it used to work as long as the number of columns and column types match.
Is this true?
The part of the error is:
java.lang.NullPointerException at org.apache.hive.hcatalog.data.schema.HCatSchema.get(HCatSchema.java:105) at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:390) at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:783) at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:98) ...
Created 02-03-2016 05:46 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Looks like if I wanted to use HCat, the column names need to match...
Created 10-27-2015 12:54 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
what's the schema on the database side?
what's the schema on the hcatalog side?
what's the sqoop query?
Created 10-28-2015 12:52 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
ORACLE:
CREATE TABLE HAJIME.SQ_TEST ( COLUMN_NUMBER NUMBER(5,0), P_SC VARCHAR2(16 CHAR), P_YMDH NUMBER(14,0) ); INSERT INTO HAJIME.SQ_TEST (P_YMDH, P_SC, COLUMN_NUMBER) VALUES (5, 't', 4); INSERT INTO HAJIME.SQ_TEST (P_YMDH, P_SC, COLUMN_NUMBER) VALUES (2, 'jlfkooakywhsc', 1); INSERT INTO HAJIME.SQ_TEST (P_YMDH, P_SC, COLUMN_NUMBER) VALUES (3, 'vp', 3);
HIVE:
create table sq_test ( column_number bigint, logacc_no string, ymdh bigint );
SQOOP COMMAND:
sqoop import --verbose --username hajime --password ***** --connect jdbc:oracle:thin:@ORACLE_SERVER:SID --query "select * from hajime.SQ_TEST WHERE \$CONDITIONS" --split-by COLUMN_NUMBER --hcatalog-table sq_test
Then I get NullPointerException with the following error:
15/10/28 00:47:40 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Caught Exception checking database column p_sc in hcatalog table.
Created 02-03-2016 02:00 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Hajime Can you post the solution you used?
Created 02-03-2016 05:46 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Looks like if I wanted to use HCat, the column names need to match...