Hive job failed using INSERT INTO TABLE with error ailed to execute tez graph.

[#1]

I try to insert the table into hdfs following by this tutorial guideline https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.1.0/using-hiveql/content/hive_create_an_externa...

by using beeline with user hdfs.

I'm done with put csv to hdfs and create external table, table but stuck at the step

'INSERT OVERWRITE TABLE ' ' SELECT * FROM ' ';

Once run this command, returning with errors below

0: jdbc:hive2://hostname:21> INSERT OVERWRITE TABLE Sales SELECT * FROM sales_text;
INFO  : Tez session hasn't been created yet. Opening session
ERROR : Failed to execute tez graph.
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1556784628398_0008 failed 2 times due to AM Container for appattempt_1556784628398_0008_000002 exited with  exitCode: -1000
For more detailed output, check the application tracking page: http://hostname:8088/cluster/app/application_1556784628398_0008 Then click on links to logs of each attempt.
Diagnostics: Application application_1556784628398_0008 initialization failed (exitCode=255) with output: main : command provided 0
main : run as user is hdfs
main : requested yarn user is hdfs
Requested user hdfs is banned

Failing this attempt. Failing the application.
    at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:699)
    at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:218)
    at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:286)
    at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:165)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:162)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1765)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1506)
    at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1303)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1170)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1165)
    at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
    at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
    at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
    at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask (state=08S01,code=1)

and did check for given url in yarnUI

Diagnostics:

Application application_1556784628398_0017 failed 2 times due to AM Container for appattempt_1556784628398_0017_000002 exited with exitCode: -1000 For more detailed output, check the application tracking page: http://hostname:8088/cluster/app/application_1556784628398_0017 Then click on links to logs of each attempt. Diagnostics: Application application_1556784628398_0017 initialization failed (exitCode=255) with output: main : command provided 0 main : run as user is hdfs main : requested yarn user is hdfs Requested user hdfs is banned Failing this attempt. Failing the application.

For the logging provided in the AttempID

Logs not available for container_e26_1556784628398_0017_02_000001. Aggregation may not be complete, Check back later or try the nodemanager at hostname:45454

Please give me any suggestions.

Thank you in advanced

[#2]

add this property in hive-site.xml

tez.am.resource.memory.mb = 2048(depends on node size)

[#3]

Unfortunately, it didn't work.

Got all the same errors

[#4]

@Banthita Limwilai

I am wondering which user you are using to execute the job?

Can you share your /etc/hadoop/conf/container-executor.cfg hdfs user should be a banned user in the previous file

Could you do the following as root user

# su - hive 

Then invoke the hive CLI, that should work

Please revert

[#5]

Here are the .ctg file

yarn.nodemanager.linux-container-executor.group=hadoop
banned.users=hdfs,yarn,mapred,bin
min.user.id=1000

According to Kerberos, Hive user doesn't has permission to execute the command.

so I ran with root instead

and using beeline not hive CLI

Thanks

[#6]

@Banthita Limwilai

Have you tried the hive user?

Can you point me to this document "According to Kerberos, Hive user doesn't has permission"

This is what I get when I true to access beeline as user hive

[hive@osaka ~]$ beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://osaka.com:2181/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
Enter username for jdbc:hive2://osaka.com:2181/default: hive
Enter password for jdbc:hive2://osaka.com:2181/default: ****
Error: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 configs from ZooKeeper (state=,code=0)
Beeline version 3.1.0.3.1.0.0-78 by Apache Hive
beeline>

HTH

[#7]

No, I mean just only my system. They not gave permission for user 'hive' for Kerberos.

And I'm not able change the permission.

Thanks

[#8]

@Banthita Limwilai

Please can you elaborate I don't seem to understand you.