question Re: Create users in hadoop/HDP 2.5 in Support Questions

Create users in hadoop/HDP 2.5

sunilreddykallu — Thu, 07 Sep 2017 22:29:41 GMT

I just created users in /home in edge node for users to work on hadoop. But users not able to run hive/Yarn jobs. My cluster is already kerberized. Is there any way to assign users to submit their jobs in edge node.

Do we need to these users in hadoop/hdfs ?

If cluster has kerberos and ranger is there any different way to do it ?

Please suggest me.

Re: Create users in hadoop/HDP 2.5

Shelton — Fri, 08 Sep 2017 01:42:49 GMT

@Sam Red

For a user to submit successfully or even connect to hive in a kerberized environment the user MUST first get a valid Kerberos ticket.

Can you users list the principals attached to their keytabs eg for user opera1

# su - opera1$ 
$ klist 
klist: No credentials cache found (filename: /tmp/krb5cc_1001)

Above output show no valid kerberos ticket ,then follow the below steps as user opera1 assuming your keytabs are in that directory and the REALM is DEV.COM

$ klist -kt /etc/security/keytabs/opera1.keytab
Keytab name: FILE:/etc/security/keytabs/opera1.keytab
KVNO              Timestamp                    Principal
---- ------------------- ------------------------------------------------------
 1              08/24/2017 18:28:26            opera1@DEV.COM
$ kinit -kt /etc/security/keytabs/opera1.keytab opera1@UGANDA.COM
$ hive

The user should be able now to run a job on the cluster !

Please let me know

Re: Create users in hadoop/HDP 2.5

sunilreddykallu — Fri, 08 Sep 2017 01:52:24 GMT

@Geoffrey Shelton Okot

Thank you again. these is the issue i am getting from beeline.

beeline> !connect jdbc:hive2://hostname.host.com:2181,hostname.host.com:2181,hostname.host.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 username password
Connected to: Apache Hive (version 1.2.1000.2.5.6.0-40)
Driver: Hive JDBC (version 1.2.1000.2.5.6.0-40)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://host> select max(_TIMESTAMP(ts)) ;
INFO  : Tez session hasn't been created yet. Opening session
ERROR : Failed to execute tez graph.
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_ failed 2 times due to AM Container for appattempt_ exited with  exitCode: -1000

Diagnostics: Application application_ID initialization failed (exitCode=255) with output: main : command provided 0
main : run as user is berlin
main : requested yarn user is berlin
User berlin not found


Failing this attempt. Failing the application.
        at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:779)
        at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:217)
        at org.apache.hadoop.hive.ql.exec.tez.TezTask.updateSession(TezTask.java:287)
        at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:166)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1745)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1491)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1289)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1156)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1151)
        at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:197)
        at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:76)
        at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:253)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1865)
        at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:264)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask (state=08S01,code=1)

Re: Create users in hadoop/HDP 2.5

Shelton — Fri, 08 Sep 2017 02:25:32 GMT

@Sam Red

When your a cluster integrated with Kerberos security then authenticated user must exist in the every node where the task runs. So create the berlin user on all the hosts and add user berlin to the hadoop group that should resolve the problem.

Please revert

Re: Create users in hadoop/HDP 2.5

sunilreddykallu — Fri, 08 Sep 2017 02:53:59 GMT

@Geoffrey Shelton Okot

Is there any way to add user in all hosts ? Please let me know

Re: Create users in hadoop/HDP 2.5

Shelton — Fri, 08 Sep 2017 03:03:48 GMT

@Sam Red

Unfortunately, you will have to use the classic way 🙂 depending on your OS adapt appropriate commands as root, below example is on centos6

# useradd user15
# passwd user15

And repeat that on all the hosts in the cluster, from the Ambari server if you created a passwordless ssh then it's easier!

# ssh root@host5
[root@host5 ~]# useradd user15 
[root@host5 ~]# passwd user15

Tedious work ..... if you have a cluster with 100 nodes!

Re: Create users in hadoop/HDP 2.5

mmullins — Fri, 08 Sep 2017 09:41:13 GMT

You need to create user directories on HDFS for each user.

$ su - hdfs

Because your cluster is kerberized you need to get a token for the hdfs user. Something like this

$ kinit -k -t /etc/security/keytabs/hdfs.service.keytab hdfs/node1.hadoop.com@HADOOP.COM

Now you can create an HDFS directory for your user.

$ hadoop fs -mkdir /user/berlin
$ hadoop fs -chown berlin /user/berlin

If you're querying Hive, you could use a desktop tool like Aginity Workbench, which supports Kerberos connections.

Re: Create users in hadoop/HDP 2.5

bkhanna — Fri, 08 Sep 2017 13:54:12 GMT

Hi,

Could you please execute your query with beeline --verbose=true.

What happens if the query is run from Hive Cli?

Re: Create users in hadoop/HDP 2.5

rvillanueva — Thu, 15 Aug 2019 09:40:21 GMT

@Geoffrey Shelton Okot

What about if the cluster is not using kerberos (eg. hadoop.security.authentication=local)?

Re: Create users in hadoop/HDP 2.5

asmarz — Tue, 04 Feb 2020 22:10:21 GMT

Hello 🙂 I have the same issue!

I have integrated the edge node with Active directory users could connect and submit theirs jobs to yarn before enabling Kerberos on the cluster.

Actually I have used samba on edge node to create users folders and get information about users

Now I configured the Kerberos and so I am getting the same error user1 not found , user1 is in AD

should I now add this user with normal command add user on alll nodes ? How could it be as AD user and not local one ? I did not configured samba on others nodes may I do it ?

thanks a lot in advance

Re: Create users in hadoop/HDP 2.5

ask_bill_brooks — Tue, 04 Feb 2020 23:54:09 GMT

@asmarz,

As this is an older thread which was previously marked 'Solved', you would have a better chance of receiving a resolution by starting a new thread. This will also provide the opportunity to provide details specific to your environment (for example, what happened once you added the affected user accounts with "normal" command add user on all nodes) that could aid others in providing a more relevant, accurate answer to your question.