Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

workflow ERROR hive.ql.metadata.Hive - MetaException(message:java.lang.NullPointerException)

workflow ERROR hive.ql.metadata.Hive - MetaException(message:java.lang.NullPointerException)

Contributor

hi:

 

with hue,I create a new user and group(bi:bi),and with sentry,I had set all privilige on datbase bi_system.

 

when I run a workflow,the code like this:

use bi_system;

create table if not exists hive_job_test1(
country string,
clicks int,
conversions int
)partitioned by(part string);

insert overwrite table hive_job_test1 partition(part='${out}')
select country,clicks,conversions from hive_job_test where part='${in}';

and then,the mapreduce work,but the last move task not work.the error :

6596 [main] INFO  org.apache.hadoop.hive.ql.exec.Utilities  - No plan file found: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-mr-10004/9f5a5eb1-b1ca-4007-b495-9eb734e93047/reduce.xml
  7178 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=getSplits from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat>
  7226 [main] INFO  org.apache.hadoop.hive.ql.io.CombineHiveInputFormat  - CombineHiveInputSplit creating pool for hdfs://master-71:8020/user/hive/warehouse/bi_system.db/hive_job_test/part=2014-09-20; using filter path hdfs://master-71:8020/user/hive/warehouse/bi_system.db/hive_job_test/part=2014-09-20
  7279 [main] INFO  org.apache.hadoop.hive.ql.io.CombineHiveInputFormat  - number of splits 1
  7279 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=getSplits start=1430992754318 end=1430992754419 duration=101 from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat>
  8349 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Starting Job = job_1430989734038_0014, Tracking URL = http://master-71:8088/proxy/application_1430989734038_0014/
  8350 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Kill Command = /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop job  -kill job_1430989734038_0014
  22776 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
  22833 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - 2015-05-07 17:59:29,972 Stage-1 map = 0%,  reduce = 0%
  31466 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - 2015-05-07 17:59:38,605 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.35 sec
  32515 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - MapReduce Total cumulative CPU time: 1 seconds 350 msec
  32561 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Ended Job = job_1430989734038_0014
  32579 [main] INFO  org.apache.hadoop.hive.ql.exec.FileSinkOperator  - Moving tmp dir: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/_tmp.-ext-10002 to: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10002
  32599 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.CONDITION.Stage-7 from=org.apache.hadoop.hive.ql.Driver>
  32604 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-4 is selected by condition resolver.
  32604 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-3 is filtered out by condition resolver.
  32605 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-5 is filtered out by condition resolver.
  32605 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.MOVE.Stage-4 from=org.apache.hadoop.hive.ql.Driver>
  32607 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Moving data to: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10000 from hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10002
  Heart beat
  32631 [main] INFO  hive.ql.metadata.Hive  - Replacing src:hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10002;dest: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10000;Status:true
  32731 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.MOVE.Stage-0 from=org.apache.hadoop.hive.ql.Driver>
  32733 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Loading data to table bi_system.hive_job_test1 partition (part=2015-05-01) from hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10000
  32765 [main] INFO  org.apache.hadoop.hive.ql.exec.MoveTask  - Partition is: {part=2015-05-01}
  32835 [main] INFO  hive.ql.metadata.Hive  - Replacing src:hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10000;dest: hdfs://master-71:8020/user/hive/warehouse/bi_system.db/hive_job_test1/part=2015-05-01;Status:true
  32910 [main] ERROR hive.ql.metadata.Hive  - MetaException(message:java.lang.NullPointerException)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41079)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41038)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result.read(ThriftHiveMetastore.java:40961)
  at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_append_partition_with_environment_context(ThriftHiveMetastore.java:1355)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.append_partition_with_environment_context(ThriftHiveMetastore.java:1339)
  at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:494)
  at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:488)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
  at com.sun.proxy.$Proxy21.appendPartition(Unknown Source)
  at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1647)
  at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1279)
  at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:403)
  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
  at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
  at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
  at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
  at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
  at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
  at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
  at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
  at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
  at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
  at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
  at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
  at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)
  at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
  at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
  
  32924 [main] ERROR org.apache.hadoop.hive.ql.exec.Task  - Failed with exception MetaException(message:java.lang.NullPointerException)
  org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.lang.NullPointerException)
  at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1679)
  at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1279)
  at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:403)
  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
  at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
  at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
  at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
  at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
  at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
  at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
  at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
  at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
  at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
  at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
  at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
  at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)
  at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
  at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
  Caused by: MetaException(message:java.lang.NullPointerException)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41079)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41038)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result.read(ThriftHiveMetastore.java:40961)
  at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_append_partition_with_environment_context(ThriftHiveMetastore.java:1355)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.append_partition_with_environment_context(ThriftHiveMetastore.java:1339)
  at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:494)
  at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:488)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
  at com.sun.proxy.$Proxy21.appendPartition(Unknown Source)
  at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1647)
  ... 35 more
  
  32924 [main] ERROR org.apache.hadoop.hive.ql.Driver  - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
  32924 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=Driver.execute start=1430992753359 end=1430992780064 duration=26705 from=org.apache.hadoop.hive.ql.Driver>
  32924 [main] INFO  org.apache.hadoop.hive.ql.Driver  - MapReduce Jobs Launched:
  32925 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Stage-Stage-1: Map: 1   Cumulative CPU: 1.35 sec   HDFS Read: 2472 HDFS Write: 1774 SUCCESS
  32925 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Total MapReduce CPU Time Spent: 1 seconds 350 msec
  32925 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
  33026 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=releaseLocks start=1430992780065 end=1430992780166 duration=101 from=org.apache.hadoop.hive.ql.Driver>
  33036 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
  33036 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=releaseLocks start=1430992780176 end=1430992780176 duration=0 from=org.apache.hadoop.hive.ql.Driver>
  
  <<< Invocation of Hive command completed <<<
  
  Hadoop Job IDs executed by Hive: job_1430989734038_0014
  
  Intercepting System.exit(1)
  
  <<< Invocation of Main class completed <<<
  
  Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [1]
  
  Oozie Launcher failed, finishing Hadoop job gracefully
  
  Oozie Launcher, uploading action data to HDFS sequence file: hdfs://master-71:8020/user/bi/oozie-oozi/0000037-150506181648497-oozie-oozi-W/qq--hive/action-data.seq
  
  Oozie Launcher ends
              

user hue or hive run this workflow ok. just user bi.

How to fix it,please hlep me.

 

rube

 

4 REPLIES 4

Re: workflow ERROR hive.ql.metadata.Hive - MetaException(message:java.lang.NullPointerException)

This is probably a perm issue, maybe there is more info in the Yarn logs.

 

Which Hive version is it too, the error is internal to Hive and not Hue?

Re: workflow ERROR hive.ql.metadata.Hive - MetaException(message:java.lang.NullPointerException)

Contributor

1.I had installed CDH5.2.0 with kerberos,ldap and sentry.
But when I config ldap with hive, the CDH5.2.0's hive version throw "org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS initiate failed",
For this reason,I had replaced the CDH5.2.0 hive server2 jars with CDH5.2.4 hive jars,and hive and ldap works fine.(/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive/lib)

2.here is the yarn log

33170 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - 2015-05-07 17:25:18,641 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.4 sec
34213 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - MapReduce Total cumulative CPU time: 1 seconds 400 msec
34266 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Ended Job = job_1430989734038_0008
34285 [main] INFO  org.apache.hadoop.hive.ql.exec.FileSinkOperator  - Moving tmp dir: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/_tmp.-ext-10002 to: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10002
34303 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.CONDITION.Stage-7 from=org.apache.hadoop.hive.ql.Driver>
34308 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-4 is selected by condition resolver.
34308 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-3 is filtered out by condition resolver.
34308 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-5 is filtered out by condition resolver.
34308 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.MOVE.Stage-4 from=org.apache.hadoop.hive.ql.Driver>
34309 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Moving data to: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10000 from hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10002
34333 [main] INFO  hive.ql.metadata.Hive  - Replacing src:hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10002;dest: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10000;Status:true
34422 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.MOVE.Stage-0 from=org.apache.hadoop.hive.ql.Driver>
34423 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Loading data to table bi_system.hive_job_test1 partition (part=2015-05-01) from hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10000
34446 [main] INFO  org.apache.hadoop.hive.ql.exec.MoveTask  - Partition is: {part=2015-05-01}
34528 [main] INFO  hive.ql.metadata.Hive  - Replacing src:hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10000;dest: hdfs://master-71:8020/user/hive/warehouse/bi_system.db/hive_job_test1/part=2015-05-01;Status:true
34640 [main] ERROR hive.ql.metadata.Hive  - MetaException(message:java.lang.NullPointerException)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41079)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41038)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result.read(ThriftHiveMetastore.java:40961)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_append_partition_with_environment_context(ThriftHiveMetastore.java:1355)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.append_partition_with_environment_context(ThriftHiveMetastore.java:1339)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:494)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:488)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
	at com.sun.proxy.$Proxy21.appendPartition(Unknown Source)
	at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1647)
	at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1279)
	at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:403)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
	at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
	at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
	at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)
	at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
	at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)

34654 [main] ERROR org.apache.hadoop.hive.ql.exec.Task  - Failed with exception MetaException(message:java.lang.NullPointerException)
org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.lang.NullPointerException)
	at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1679)
	at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1279)
	at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:403)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
	at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
	at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
	at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)
	at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
	at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: MetaException(message:java.lang.NullPointerException)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41079)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41038)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result.read(ThriftHiveMetastore.java:40961)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_append_partition_with_environment_context(ThriftHiveMetastore.java:1355)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.append_partition_with_environment_context(ThriftHiveMetastore.java:1339)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:494)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:488)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
	at com.sun.proxy.$Proxy21.appendPartition(Unknown Source)
	at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1647)
	... 35 more

34654 [main] ERROR org.apache.hadoop.hive.ql.Driver  - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
34654 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=Driver.execute start=1430990693548 end=1430990720125 duration=26577 from=org.apache.hadoop.hive.ql.Driver>
34654 [main] INFO  org.apache.hadoop.hive.ql.Driver  - MapReduce Jobs Launched: 
34654 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Stage-Stage-1: Map: 1   Cumulative CPU: 1.4 sec   HDFS Read: 2472 HDFS Write: 1774 SUCCESS
34654 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Total MapReduce CPU Time Spent: 1 seconds 400 msec
34655 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
34745 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=releaseLocks start=1430990720126 end=1430990720216 duration=90 from=org.apache.hadoop.hive.ql.Driver>
34756 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
34756 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=releaseLocks start=1430990720227 end=1430990720227 duration=0 from=org.apache.hadoop.hive.ql.Driver>

<<< Invocation of Hive command completed <<<

 Hadoop Job IDs executed by Hive: job_1430989734038_0008

Intercepting System.exit(1)

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [1]

Oozie Launcher failed, finishing Hadoop job gracefully

Oozie Launcher, uploading action data to HDFS sequence file: hdfs://master-71:8020/user/bi/oozie-oozi/0000034-150506181648497-oozie-oozi-W/qq--hive/action-data.seq

Oozie Launcher ends



3.If I add the user bi into hive configuration sentry.metastore.service.users,the workflow works fine,I do't know why,and don't think the problem has been solved.

Aanother problem,please see point 4

4.Here is an interesting phenomenon.With beeswax, I create a table or insert data to a table,the table/data will belongs to hive:hive.
But with workflow or coordinators,the table/data will belongs to test user and supergroup,beeswax can not access the table with sql.

problem.png

 

my English is not well,please excuse...

 

 

rube

Highlighted

Re: workflow ERROR hive.ql.metadata.Hive - MetaException(message:java.lang.NullPointerException)

Contributor

Is it because hue 3.6.0 don't have hive server2 action?

Re: workflow ERROR hive.ql.metadata.Hive - MetaException(message:java.lang.NullPointerException)

Contributor

Is it because hue 3.6.0 don't have hive server2 action?


@romain wrote:

This is probably a perm issue, maybe there is more info in the Yarn logs.

 

Which Hive version is it too, the error is internal to Hive and not Hue?


 

Don't have an account?
Coming from Hortonworks? Activate your account here