Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

my hive query is failing with NPE when I am querying column with timestamp on hive partitioned orc table.

Caused by: java.lang.NullPointerException

at org.apache.hadoop.hive.ql.io.orc.ColumnStatisticsImpl$TimestampStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:795)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.getMin(RecordReaderImpl.java:2343)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicate(RecordReaderImpl.java:2366)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups(RecordReaderImpl.java:2564)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:2627)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:3060)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:3102)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.<init>(RecordReaderImpl.java:288)

at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:534)

at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$ReaderPair.<init>(OrcRawRecordMerger.java:183)

1 ACCEPTED SOLUTION

it seems that you are hitting https://issues.apache.org/jira/browse/HIVE-12349

can you try to run your query after setting hive.optimize.index.filter=false

View solution in original post

3 REPLIES 3

it seems that you are hitting https://issues.apache.org/jira/browse/HIVE-12349

can you try to run your query after setting hive.optimize.index.filter=false

@a kumar

This can be relate to Hive Jira 12349: https://issues.apache.org/jira/browse/HIVE-12349

Can you please share the query being run?

Thanks and Regards,

Sindhu

Hi I too face this issue(with tez and mr engines) now for below italized query - query to fetch a set of records from a view and insert into a table.

HDP Version : (2.3.2.0-2950) version in both dev and prod.

Hive Version : Hive 1.2.1.2.3.2.48-5

Same query worked for a week in production and 1st time i face this issue. is it due to incorrect data in Date Column?

insert overwrite table crhs_fmtrade_transient_break partition(transient_break_reporting_date, source_system)

select break_id, trade_id, reporting_date, break_type, first_seen_date, processed_date,transientbreakcloseddate, closuretype,transientbreakindicator, transientbreaktype, breakfinalsysteminvststatus, transientbreakprocesseddate, commentarysysteminvestigationstatus, transientbreakreportingdate, source_system from v_fmtrade_mxg_transient_break;

Error StackTrace (Hive on Tez):

Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.orc.ColumnStatisticsImpl$DateStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:745) at org.apache.hadoop.hive.ql.io.orc.ColumnStatisticsImpl$DateStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:665) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.getMin(RecordReaderImpl.java:304) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateProto(RecordReaderImpl.java:332) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$SargApplier.pickRowGroups(RecordReaderImpl.java:710) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups(RecordReaderImpl.java:751) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:777) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:986) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1019) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.<init>(RecordReaderImpl.java:205) at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:598) at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$ReaderPair.<init>(OrcRawRecordMerger.java:183) at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$OriginalReaderPair.<init>(OrcRawRecordMerger.java:226) at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.<init>(OrcRawRecordMerger.java:437) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1235) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1117) at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:249) ... 26 more

Error StackTrace (Hive on MR engine):

Caused by: java.lang.NullPointerException

at org.apache.hadoop.hive.ql.io.orc.ColumnStatisticsImpl$DateStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:745)

at org.apache.hadoop.hive.ql.io.orc.ColumnStatisticsImpl$DateStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:665)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.getMin(RecordReaderImpl.java:304)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateProto(RecordReaderImpl.java:332)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$SargApplier.pickRowGroups(RecordReaderImpl.java:710)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups(RecordReaderImpl.java:751)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:777)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:986)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1019)

at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.<init>(RecordReaderImpl.java:205)

at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:598)

at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.createReaderFromFile(OrcInputFormat.java:230)

at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.<init>(OrcInputFormat.java:163)

at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1108)

at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:67)

... 16 more