Support Questions
Find answers, ask questions, and share your expertise

workflow ERROR hive.ql.metadata.Hive - MetaException(message:java.lang.NullPointerException)

Contributor

hi:

 

with hue,I create a new user and group(bi:bi),and with sentry,I had set all privilige on datbase bi_system.

 

when I run a workflow,the code like this:

use bi_system;

create table if not exists hive_job_test1(
country string,
clicks int,
conversions int
)partitioned by(part string);

insert overwrite table hive_job_test1 partition(part='${out}')
select country,clicks,conversions from hive_job_test where part='${in}';

and then,the mapreduce work,but the last move task not work.the error :

6596 [main] INFO  org.apache.hadoop.hive.ql.exec.Utilities  - No plan file found: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-mr-10004/9f5a5eb1-b1ca-4007-b495-9eb734e93047/reduce.xml
  7178 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=getSplits from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat>
  7226 [main] INFO  org.apache.hadoop.hive.ql.io.CombineHiveInputFormat  - CombineHiveInputSplit creating pool for hdfs://master-71:8020/user/hive/warehouse/bi_system.db/hive_job_test/part=2014-09-20; using filter path hdfs://master-71:8020/user/hive/warehouse/bi_system.db/hive_job_test/part=2014-09-20
  7279 [main] INFO  org.apache.hadoop.hive.ql.io.CombineHiveInputFormat  - number of splits 1
  7279 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=getSplits start=1430992754318 end=1430992754419 duration=101 from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat>
  8349 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Starting Job = job_1430989734038_0014, Tracking URL = http://master-71:8088/proxy/application_1430989734038_0014/
  8350 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Kill Command = /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop job  -kill job_1430989734038_0014
  22776 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
  22833 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - 2015-05-07 17:59:29,972 Stage-1 map = 0%,  reduce = 0%
  31466 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - 2015-05-07 17:59:38,605 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.35 sec
  32515 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - MapReduce Total cumulative CPU time: 1 seconds 350 msec
  32561 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Ended Job = job_1430989734038_0014
  32579 [main] INFO  org.apache.hadoop.hive.ql.exec.FileSinkOperator  - Moving tmp dir: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/_tmp.-ext-10002 to: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10002
  32599 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.CONDITION.Stage-7 from=org.apache.hadoop.hive.ql.Driver>
  32604 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-4 is selected by condition resolver.
  32604 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-3 is filtered out by condition resolver.
  32605 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-5 is filtered out by condition resolver.
  32605 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.MOVE.Stage-4 from=org.apache.hadoop.hive.ql.Driver>
  32607 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Moving data to: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10000 from hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10002
  Heart beat
  32631 [main] INFO  hive.ql.metadata.Hive  - Replacing src:hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10002;dest: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10000;Status:true
  32731 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.MOVE.Stage-0 from=org.apache.hadoop.hive.ql.Driver>
  32733 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Loading data to table bi_system.hive_job_test1 partition (part=2015-05-01) from hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10000
  32765 [main] INFO  org.apache.hadoop.hive.ql.exec.MoveTask  - Partition is: {part=2015-05-01}
  32835 [main] INFO  hive.ql.metadata.Hive  - Replacing src:hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-59-11_762_709570366507990110-1/-ext-10000;dest: hdfs://master-71:8020/user/hive/warehouse/bi_system.db/hive_job_test1/part=2015-05-01;Status:true
  32910 [main] ERROR hive.ql.metadata.Hive  - MetaException(message:java.lang.NullPointerException)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41079)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41038)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result.read(ThriftHiveMetastore.java:40961)
  at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_append_partition_with_environment_context(ThriftHiveMetastore.java:1355)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.append_partition_with_environment_context(ThriftHiveMetastore.java:1339)
  at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:494)
  at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:488)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
  at com.sun.proxy.$Proxy21.appendPartition(Unknown Source)
  at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1647)
  at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1279)
  at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:403)
  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
  at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
  at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
  at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
  at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
  at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
  at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
  at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
  at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
  at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
  at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
  at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
  at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)
  at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
  at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
  
  32924 [main] ERROR org.apache.hadoop.hive.ql.exec.Task  - Failed with exception MetaException(message:java.lang.NullPointerException)
  org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.lang.NullPointerException)
  at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1679)
  at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1279)
  at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:403)
  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
  at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
  at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
  at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
  at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
  at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
  at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
  at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
  at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
  at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
  at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
  at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
  at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
  at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
  at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)
  at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)
  at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
  at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
  at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
  at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
  Caused by: MetaException(message:java.lang.NullPointerException)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41079)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41038)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result.read(ThriftHiveMetastore.java:40961)
  at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_append_partition_with_environment_context(ThriftHiveMetastore.java:1355)
  at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.append_partition_with_environment_context(ThriftHiveMetastore.java:1339)
  at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:494)
  at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:488)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
  at com.sun.proxy.$Proxy21.appendPartition(Unknown Source)
  at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1647)
  ... 35 more
  
  32924 [main] ERROR org.apache.hadoop.hive.ql.Driver  - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
  32924 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=Driver.execute start=1430992753359 end=1430992780064 duration=26705 from=org.apache.hadoop.hive.ql.Driver>
  32924 [main] INFO  org.apache.hadoop.hive.ql.Driver  - MapReduce Jobs Launched:
  32925 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Stage-Stage-1: Map: 1   Cumulative CPU: 1.35 sec   HDFS Read: 2472 HDFS Write: 1774 SUCCESS
  32925 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Total MapReduce CPU Time Spent: 1 seconds 350 msec
  32925 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
  33026 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=releaseLocks start=1430992780065 end=1430992780166 duration=101 from=org.apache.hadoop.hive.ql.Driver>
  33036 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
  33036 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=releaseLocks start=1430992780176 end=1430992780176 duration=0 from=org.apache.hadoop.hive.ql.Driver>
  
  <<< Invocation of Hive command completed <<<
  
  Hadoop Job IDs executed by Hive: job_1430989734038_0014
  
  Intercepting System.exit(1)
  
  <<< Invocation of Main class completed <<<
  
  Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [1]
  
  Oozie Launcher failed, finishing Hadoop job gracefully
  
  Oozie Launcher, uploading action data to HDFS sequence file: hdfs://master-71:8020/user/bi/oozie-oozi/0000037-150506181648497-oozie-oozi-W/qq--hive/action-data.seq
  
  Oozie Launcher ends
              

user hue or hive run this workflow ok. just user bi.

How to fix it,please hlep me.

 

rube

 

4 REPLIES 4

Re: workflow ERROR hive.ql.metadata.Hive - MetaException(message:java.lang.NullPointerException)

This is probably a perm issue, maybe there is more info in the Yarn logs.

 

Which Hive version is it too, the error is internal to Hive and not Hue?

Re: workflow ERROR hive.ql.metadata.Hive - MetaException(message:java.lang.NullPointerException)

Contributor

1.I had installed CDH5.2.0 with kerberos,ldap and sentry.
But when I config ldap with hive, the CDH5.2.0's hive version throw "org.apache.thrift.transport.TTransportException: Peer indicated failure: GSS initiate failed",
For this reason,I had replaced the CDH5.2.0 hive server2 jars with CDH5.2.4 hive jars,and hive and ldap works fine.(/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive/lib)

2.here is the yarn log

33170 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - 2015-05-07 17:25:18,641 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.4 sec
34213 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - MapReduce Total cumulative CPU time: 1 seconds 400 msec
34266 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Ended Job = job_1430989734038_0008
34285 [main] INFO  org.apache.hadoop.hive.ql.exec.FileSinkOperator  - Moving tmp dir: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/_tmp.-ext-10002 to: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10002
34303 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.CONDITION.Stage-7 from=org.apache.hadoop.hive.ql.Driver>
34308 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-4 is selected by condition resolver.
34308 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-3 is filtered out by condition resolver.
34308 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Stage-5 is filtered out by condition resolver.
34308 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.MOVE.Stage-4 from=org.apache.hadoop.hive.ql.Driver>
34309 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Moving data to: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10000 from hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10002
34333 [main] INFO  hive.ql.metadata.Hive  - Replacing src:hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10002;dest: hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10000;Status:true
34422 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=task.MOVE.Stage-0 from=org.apache.hadoop.hive.ql.Driver>
34423 [main] INFO  org.apache.hadoop.hive.ql.exec.Task  - Loading data to table bi_system.hive_job_test1 partition (part=2015-05-01) from hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10000
34446 [main] INFO  org.apache.hadoop.hive.ql.exec.MoveTask  - Partition is: {part=2015-05-01}
34528 [main] INFO  hive.ql.metadata.Hive  - Replacing src:hdfs://master-71:8020/tmp/hive-bi/hive_2015-05-07_17-24-51_735_8642627058175283778-1/-ext-10000;dest: hdfs://master-71:8020/user/hive/warehouse/bi_system.db/hive_job_test1/part=2015-05-01;Status:true
34640 [main] ERROR hive.ql.metadata.Hive  - MetaException(message:java.lang.NullPointerException)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41079)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41038)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result.read(ThriftHiveMetastore.java:40961)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_append_partition_with_environment_context(ThriftHiveMetastore.java:1355)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.append_partition_with_environment_context(ThriftHiveMetastore.java:1339)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:494)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:488)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
	at com.sun.proxy.$Proxy21.appendPartition(Unknown Source)
	at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1647)
	at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1279)
	at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:403)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
	at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
	at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
	at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)
	at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
	at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)

34654 [main] ERROR org.apache.hadoop.hive.ql.exec.Task  - Failed with exception MetaException(message:java.lang.NullPointerException)
org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.lang.NullPointerException)
	at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1679)
	at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1279)
	at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:403)
	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1516)
	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1283)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1101)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:924)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:914)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:269)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:221)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:431)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:367)
	at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:464)
	at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:474)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:694)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:633)
	at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:323)
	at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:284)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
	at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:227)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: MetaException(message:java.lang.NullPointerException)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41079)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result$append_partition_with_environment_context_resultStandardScheme.read(ThriftHiveMetastore.java:41038)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$append_partition_with_environment_context_result.read(ThriftHiveMetastore.java:40961)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_append_partition_with_environment_context(ThriftHiveMetastore.java:1355)
	at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.append_partition_with_environment_context(ThriftHiveMetastore.java:1339)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:494)
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.appendPartition(HiveMetaStoreClient.java:488)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
	at com.sun.proxy.$Proxy21.appendPartition(Unknown Source)
	at org.apache.hadoop.hive.ql.metadata.Hive.getPartition(Hive.java:1647)
	... 35 more

34654 [main] ERROR org.apache.hadoop.hive.ql.Driver  - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
34654 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=Driver.execute start=1430990693548 end=1430990720125 duration=26577 from=org.apache.hadoop.hive.ql.Driver>
34654 [main] INFO  org.apache.hadoop.hive.ql.Driver  - MapReduce Jobs Launched: 
34654 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Stage-Stage-1: Map: 1   Cumulative CPU: 1.4 sec   HDFS Read: 2472 HDFS Write: 1774 SUCCESS
34654 [main] INFO  org.apache.hadoop.hive.ql.Driver  - Total MapReduce CPU Time Spent: 1 seconds 400 msec
34655 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
34745 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=releaseLocks start=1430990720126 end=1430990720216 duration=90 from=org.apache.hadoop.hive.ql.Driver>
34756 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
34756 [main] INFO  org.apache.hadoop.hive.ql.log.PerfLogger  - </PERFLOG method=releaseLocks start=1430990720227 end=1430990720227 duration=0 from=org.apache.hadoop.hive.ql.Driver>

<<< Invocation of Hive command completed <<<

 Hadoop Job IDs executed by Hive: job_1430989734038_0008

Intercepting System.exit(1)

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [1]

Oozie Launcher failed, finishing Hadoop job gracefully

Oozie Launcher, uploading action data to HDFS sequence file: hdfs://master-71:8020/user/bi/oozie-oozi/0000034-150506181648497-oozie-oozi-W/qq--hive/action-data.seq

Oozie Launcher ends



3.If I add the user bi into hive configuration sentry.metastore.service.users,the workflow works fine,I do't know why,and don't think the problem has been solved.

Aanother problem,please see point 4

4.Here is an interesting phenomenon.With beeswax, I create a table or insert data to a table,the table/data will belongs to hive:hive.
But with workflow or coordinators,the table/data will belongs to test user and supergroup,beeswax can not access the table with sql.

problem.png

 

my English is not well,please excuse...

 

 

rube

Re: workflow ERROR hive.ql.metadata.Hive - MetaException(message:java.lang.NullPointerException)

Contributor

Is it because hue 3.6.0 don't have hive server2 action?

Re: workflow ERROR hive.ql.metadata.Hive - MetaException(message:java.lang.NullPointerException)

Contributor

Is it because hue 3.6.0 don't have hive server2 action?


@romain wrote:

This is probably a perm issue, maybe there is more info in the Yarn logs.

 

Which Hive version is it too, the error is internal to Hive and not Hue?