Member since
06-09-2016
6
Posts
0
Kudos Received
0
Solutions
12-10-2016
07:41 PM
2 Kudos
looks you are having this problem on kerberized cluster and user dont have access on kafka topic, please try to set kafka topic acl using following link. https://community.hortonworks.com/articles/58370/kafka-topic-creation-and-acl-configuration-for-atl.html
... View more
06-17-2016
03:03 PM
also Spark history UI has a lot of cool information
... View more
06-17-2016
02:45 PM
increase heap space to be at least 471 MB. try to set VM options to -Xmx512m.
... View more
03-13-2017
01:22 PM
Hi I too face this issue(with tez and mr engines) now for below italized query - query to fetch a set of records from a view and insert into a table. HDP Version : (2.3.2.0-2950) version in both dev and prod. Hive Version : Hive 1.2.1.2.3.2.48-5 Same query worked for a week in production and 1st time i face this issue. is it due to incorrect data in Date Column? insert overwrite table crhs_fmtrade_transient_break partition(transient_break_reporting_date, source_system) select break_id, trade_id, reporting_date, break_type, first_seen_date, processed_date,transientbreakcloseddate, closuretype,transientbreakindicator, transientbreaktype, breakfinalsysteminvststatus, transientbreakprocesseddate, commentarysysteminvestigationstatus, transientbreakreportingdate, source_system from v_fmtrade_mxg_transient_break; Error StackTrace (Hive on Tez): Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.orc.ColumnStatisticsImpl$DateStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:745) at org.apache.hadoop.hive.ql.io.orc.ColumnStatisticsImpl$DateStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:665) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.getMin(RecordReaderImpl.java:304) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateProto(RecordReaderImpl.java:332) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$SargApplier.pickRowGroups(RecordReaderImpl.java:710) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups(RecordReaderImpl.java:751) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:777) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:986) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1019) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.<init>(RecordReaderImpl.java:205) at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:598) at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$ReaderPair.<init>(OrcRawRecordMerger.java:183) at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$OriginalReaderPair.<init>(OrcRawRecordMerger.java:226) at org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.<init>(OrcRawRecordMerger.java:437) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1235) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1117) at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:249) ... 26 more Error StackTrace (Hive on MR engine): Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.orc.ColumnStatisticsImpl$DateStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:745) at org.apache.hadoop.hive.ql.io.orc.ColumnStatisticsImpl$DateStatisticsImpl.getMinimum(ColumnStatisticsImpl.java:665) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.getMin(RecordReaderImpl.java:304) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.evaluatePredicateProto(RecordReaderImpl.java:332) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$SargApplier.pickRowGroups(RecordReaderImpl.java:710) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.pickRowGroups(RecordReaderImpl.java:751) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:777) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:986) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:1019) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.<init>(RecordReaderImpl.java:205) at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:598) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.createReaderFromFile(OrcInputFormat.java:230) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.<init>(OrcInputFormat.java:163) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1108) at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:67) ... 16 more
... View more
09-05-2018
05:54 PM
you are not getting the desired result as your compaction has failed. please check the yarn log to understand what might have gone wrong.
... View more