question java.lang.NullPointerException at org.apache.hive.hcatalog.data.schema.HCatSchema.get in Archives of Support Questions (Read Only)

java.lang.NullPointerException at org.apache.hive.hcatalog.data.schema.HCatSchema.get

hosako — Tue, 27 Oct 2015 15:35:36 GMT

HDP version: 2.3.0 upgraded from 2.2.4.2

A user reported his sqoop command fails if database column name and HCatalog column name are different. If he uses same column names for both side, it works. But he thinks it used to work as long as the number of columns and column types match.

Is this true?

The part of the error is:

java.lang.NullPointerException at org.apache.hive.hcatalog.data.schema.HCatSchema.get(HCatSchema.java:105) at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:390) at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:783) at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:98) ...

Re: java.lang.NullPointerException at org.apache.hive.hcatalog.data.schema.HCatSchema.get

aervits — Tue, 27 Oct 2015 19:54:33 GMT

what's the schema on the database side?

what's the schema on the hcatalog side?

what's the sqoop query?

Re: java.lang.NullPointerException at org.apache.hive.hcatalog.data.schema.HCatSchema.get

hosako — Wed, 28 Oct 2015 07:52:14 GMT

ORACLE:

CREATE TABLE HAJIME.SQ_TEST
(
  COLUMN_NUMBER                  NUMBER(5,0),
  P_SC                           VARCHAR2(16 CHAR),
  P_YMDH                         NUMBER(14,0)
);
INSERT INTO HAJIME.SQ_TEST (P_YMDH, P_SC, COLUMN_NUMBER) VALUES (5, 't', 4);
INSERT INTO HAJIME.SQ_TEST (P_YMDH, P_SC, COLUMN_NUMBER) VALUES (2, 'jlfkooakywhsc', 1);
INSERT INTO HAJIME.SQ_TEST (P_YMDH, P_SC, COLUMN_NUMBER) VALUES (3, 'vp', 3);

HIVE:

create table sq_test (
 column_number       bigint,
 logacc_no           string,
 ymdh                bigint
);

SQOOP COMMAND:

sqoop import --verbose --username hajime --password ***** --connect jdbc:oracle:thin:@ORACLE_SERVER:SID --query "select * from hajime.SQ_TEST WHERE \$CONDITIONS" --split-by COLUMN_NUMBER --hcatalog-table sq_test

Then I get NullPointerException with the following error:

15/10/28 00:47:40 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Caught Exception checking database column p_sc in hcatalog table.

Re: java.lang.NullPointerException at org.apache.hive.hcatalog.data.schema.HCatSchema.get

aervits — Wed, 03 Feb 2016 10:00:01 GMT

@Hajime Can you post the solution you used?

Re: java.lang.NullPointerException at org.apache.hive.hcatalog.data.schema.HCatSchema.get

hosako — Thu, 04 Feb 2016 01:46:33 GMT

Looks like if I wanted to use HCat, the column names need to match...