question Re: [Closed] : How to create user in HDFS in Archives of Support Questions (Read Only)

[Closed] : How to create user in HDFS

pateljay — Thu, 24 May 2018 18:40:39 GMT

I have read many documents and implements many solutions. I have made directory as `<user_name>` by following commands and gave ownership and permission to the folder as HDFS user.

[root@sandbox ~]# su hdfs
[hdfs@sandbox root]$ hdfs dfs -mkdir /user/tempuser
[hdfs@sandbox root]$ hdfs dfs -chown tempuser:hdfs /user/tempuser
[hdfs@sandbox root]$ hdfs dfs -chmod 700 /user/tempuser
[hdfs@sandbox root]$ su tempuser
su: user tempuser does not exist
[hdfs@sandbox root]$

But is throwing error as user <username> does not exist.
Please help me to resolve it.

Regards,

Jay.

Re: [Closed] : How to create user in HDFS

Shelton — Thu, 24 May 2018 19:18:55 GMT

@JAy PaTel

The above code snippet is for creating a user's home directory in HDFS.

You will need to create the local Linux user, to do so a root or if you have sudo privileges do the following

# useradd  tempuser

To set the password, you will be prompted twice

# passwd tempuser

Using sudo privileges

$ sudo useradd tempuser

See aabove

$ sudo  passwd tempuser

This will also create a user home on the local Linux box in /home/tempuser this is different from the hdfs user home /user/tempuser with MUST exst if the tempuser is to run and hive queries etc

Hope that helps

Re: [Closed] : How to create user in HDFS

pateljay — Thu, 24 May 2018 19:53:06 GMT

@Geoffrey Shelton Okot

Is it for CentOS? Because I am using Hortonworks Sandbox on Oracle Virtual Machine.

Re: [Closed] : How to create user in HDFS

Shelton — Thu, 24 May 2018 20:30:13 GMT

@JAy PaTel

Yes that okay the same commands will work

Re: [Closed] : How to create user in HDFS

mokkan — Thu, 24 May 2018 20:39:45 GMT

I'm just curious. Why do hive jobs need the home directory to run the job?

Re: [Closed] : How to create user in HDFS

Shelton — Thu, 24 May 2018 20:54:49 GMT

@Mokkan Mok

Hive jobs need a temporary working directory where hiveJars used during the execution are temporarily stored. See example

$ hdfs dfs -ls /user/hive
Found 3 items
drwx------   - hive hdfs          0 2018-04-25 20:00 /user/hive/.Trash
drwxr-xr-x   - hive hdfs          0 2018-04-13 23:04 /user/hive/.hiveJars
-rw-r--r--   3 hive hdfs        642 2018-05-24 08:43 /user/hive/derby.log
$ hdfs dfs -ls /user/hive/.hiveJars
Found 1 items
-rw-r--r--   3 hive hdfs   22006904 2018-04-13 23:04 /user/hive/.hiveJars/hive-exec-1.2.1000.2.6.2.0-205-79292bba9a3e076ad6d7a33c604b892fa0d45f6f60ae07a507e5e659a297f665.jar

Hope that helps explain

Re: [Closed] : How to create user in HDFS

Shelton — Thu, 24 May 2018 20:55:54 GMT

@JAy PaTel

If you found this answer addressed your question, please take a moment to log in and click the "accept" link on the answer.

Re: [Closed] : How to create user in HDFS

pateljay — Fri, 25 May 2018 15:17:52 GMT

Thank you @Geoffrey Shelton Okot It helps.

Actually, I was thinking it will create a user in CentOS only. But It has created a user in HDFS as well.