question Re: insert in HIVE failing in Archives of Support Questions (Read Only)

insert in HIVE failing

aliyesami — Thu, 05 May 2016 02:07:49 GMT

I am connected to HIVE using hadoop user which I created as Admin but I am still getting errors while trying to insert .

0: jdbc:hive2://SecondaryMaster:10000> INSERT INTO customers (id, firstname, lastname) VALUES (1, 'John', 'Smith'); INFO : Tez session hasn't been created yet. Opening session ERROR : Failed to execute tez graph. org.apache.hadoop.security.AccessControlException: Permission denied: user=hadoop, access=WRITE, inode="/user/hadoop":hdfs:hdfs:drwxr-xr-x

Re: insert in HIVE failing

sunile_manjee — Thu, 05 May 2016 02:08:04 GMT

@Sami Ahmad you are inserting data using user hadoop into directory /user/hadoop. to fix this provide user hadoop permission to write to file directory /user/hadoop. You can also simple add user hadoop to linux group hdfs. then all permissions group hdfs has will be inheritried by users within the group. let me know if you need more details.

Re: insert in HIVE failing

sunile_manjee — Thu, 05 May 2016 02:11:41 GMT

@Sami Ahmad you can change the owner of /user/hadoop folder by issuing this command

hdfs dfs -chown -R hadoop /user/hadoop

or hadoop fs -chown -R hadoop /user/hadoop

Do the above command as user hdfs.

Re: insert in HIVE failing

Jonathan_M_Maes — Thu, 05 May 2016 02:19:41 GMT

@Sami Ahmad, The permissions on your /user/hadoop HDFS directory are incorrect. You need to run this command to change the ownership of the HDFS directory:

hdfs dfs -chown hadoop:hdfs /user/hadoop

The error message shows you the current permissions on the /user/hadoop HDFS directory along with the type of access the process is trying to perform. Changing the /user/hadoop HDFS directory's owner using the above command should get you going again.

ERROR : Failed to execute tez graph. org.apache.hadoop.security.AccessControlException: Permission denied: user=hadoop, access=WRITE, inode="/user/hadoop":hdfs:hdfs:drwxr-xr-x

Re: insert in HIVE failing

aliyesami — Thu, 05 May 2016 03:01:19 GMT

but why its accessing /user/hadoop ? I don't even have this directory

[hdfs@ip-172-31-11-180 ~]$ hdfs dfs -ls /user Found 11 items -rw-r--r-- 3 hdfs hdfs 124 2016-04-28 17:42 /user/a.log drwx------ - accumulo hdfs 0 2016-04-28 14:57 /user/accumulo drwxrwx--- - ambari-qa hdfs 0 2016-05-04 11:53 /user/ambari-qa -rw-r--r-- 3 hdfs hdfs 124 2016-04-28 17:43 /user/b.log drwxr-xr-x - hcat hdfs 0 2016-04-28 14:57 /user/hcat drwxr-xr-x - hdfs hdfs 0 2016-05-04 12:02 /user/hdfs drwx------ - hive hdfs 0 2016-05-04 15:39 /user/hive drwxrwxr-x - oozie hdfs 0 2016-04-28 14:57 /user/oozie drwxrwxr-x - spark hdfs 0 2016-04-28 14:55 /user/spark [hdfs@ip-172-31-11-180 ~]$

Re: insert in HIVE failing

aliyesami — Thu, 05 May 2016 03:07:49 GMT

I saw this on web and after the following change the error has also changed .

"For me it was solved by changing the address that the service was listening on in /etc/hadoop/conf/core-sitse.xml. Mine was previously listening on "localhost:8020." (127.0.0.1)

<configuration> <property> <name>fs.defaultFS</name> <value>hdfs://0.0.0.0:8020</value> </property>

&& then bounce the service with hadoop "

NOW THE ERROR I GET IS :

0: jdbc:hive2://SecondaryMaster:10000> INSERT INTO customers (id, firstname, lastname) VALUES (1, 'John', 'Smith'); INFO : Tez session hasn't been created yet. Opening session ERROR : Failed to execute tez graph. org.apache.hadoop.security.AccessControlException: Permission denied: user=INSERT INTO customers (id, firstname, lastnaALUES (1, 'John', 'Smith');, access=WRITE, inode="/user/INSERT INTO customers (id, firstname, lastname) VALUES (1, 'JohSmith');":hdfs:hdfs:drwxr-xr-x

Re: insert in HIVE failing

aliyesami — Thu, 05 May 2016 03:15:57 GMT

also shall I put the change that I made in core-site.xml file back ?

Re: insert in HIVE failing

sunile_manjee — Thu, 05 May 2016 03:29:03 GMT

@Sami Ahmad Why are you change the core-site.xml? you need to update the permissions on the directory which should solve you issue as i have identified. it is trying to access that directory as it may be the default directory based on what is set on your hive.metastore.warehouse.dir. I suggest you update the file permissions and revert any changes on core-site.xml. Post what happens after you have done this.

Re: insert in HIVE failing

aliyesami — Thu, 05 May 2016 03:49:07 GMT

hi Manjee how can I change the permission of a directory that does not exist?

Re: insert in HIVE failing

aliyesami — Thu, 05 May 2016 03:59:36 GMT

[hdfs@ip-172-31-11-180 ~]$ hdfs dfs -chown -R hadoop /user/hadoop chown: `/user/hadoop': No such file or directory [hdfs@ip-172-31-11-180 ~]$

Re: insert in HIVE failing

manish1 — Thu, 05 May 2016 04:08:53 GMT

@Sami Ahmad

Please create home directory for hadoop user in HDFS first. It seems it does not exist. Please run following commands in the given sequence and try again running your hive command:

su - hdfs
hdfs dfs -mkdir /user/hadoop
hdfs dfs -chown -R hadoop /user/hadoop
exit

Re: insert in HIVE failing

aliyesami — Thu, 05 May 2016 04:28:13 GMT

I created the directory and granted permissions but now it just hangs . all services are up except knox.

[hdfs@ip-172-31-11-180 ~]$ beeline

WARNING: Use "yarn jar" to launch YARN applications. Beeline version 1.2.1.2.3.4.0-3485 by Apache Hive beeline> !connect jdbc:hive2://SecondaryMaster:10000 Connecting to jdbc:hive2://SecondaryMaster:10000 Enter username for jdbc:hive2://SecondaryMaster:10000: hadoop

Enter password for jdbc:hive2://SecondaryMaster:10000: ******

Connected to: Apache Hive (version 1.2.1.2.3.4.0-3485) Driver: Hive JDBC (version 1.2.1.2.3.4.0-3485) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://SecondaryMaster:10000> INSERT INTO customers (id, firstname, lastname) VALUES (1, 'John', 'Smith'); INFO : Tez session hasn't been created yet. Opening session INFO :

Tez session hasn't been created yet. Opening session INFO :

Tez session hasn't been created yet. Opening session

Re: insert in HIVE failing

aliyesami — Thu, 05 May 2016 04:33:17 GMT

it has created the HIVE file in the newly created directory so its doing something

[hdfs@ip-172-31-11-180 ~]$ hdfs dfs -ls /user

drwxr-xr-x - hadoop hdfs 0 2016-05-04 17:14 /user/hadoop

[hdfs@ip-172-31-11-180 ~]$ hdfs dfs -ls /user/hadoop Found 1 items drwxr-xr-x - hadoop hdfs 0 2016-05-04 17:14 /user/hadoop/.hiveJars

Re: insert in HIVE failing

manish1 — Thu, 05 May 2016 04:42:38 GMT

Please exit out from hdfs user also before running the query. It seems you ran it using hdfs user instead of hadoop user.

Re: insert in HIVE failing

aliyesami — Thu, 05 May 2016 09:55:42 GMT

hadoop user i used in the connect command was a user i created in Ambari , does it already get created on Linux?

i was running hdfs commands under hdfs and hive commands under hive Linux account.

please advise me on how to properly run each application .should i use a common account or indivisual accounts for each application?

Re: insert in HIVE failing

aliyesami — Thu, 05 May 2016 21:45:50 GMT

I need help , HIVE still not working , INSERT command just hangs

[root@ip-172-31-11-180 ~]# su - hive [hive@ip-172-31-11-180 ~]$ id uid=501(hive) gid=502(hadoop) groups=502(hadoop) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 [hive@ip-172-31-11-180 ~]$ beeline WARNING: Use "yarn jar" to launch YARN applications. Beeline version 1.2.1.2.3.4.0-3485 by Apache Hive beeline> !connect jdbc:hive2://SecondaryMaster:10000 Connecting to jdbc:hive2://SecondaryMaster:10000 Enter username for jdbc:hive2://SecondaryMaster:10000: hadoop Enter password for jdbc:hive2://SecondaryMaster:10000: ****** Connected to: Apache Hive (version 1.2.1.2.3.4.0-3485) Driver: Hive JDBC (version 1.2.1.2.3.4.0-3485) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://SecondaryMaster:10000> INSERT INTO person (id, firstname, lastname) VALUES (1, 'John', 'Smith'); INFO : Tez session hasn't been created yet. Opening session

Re: insert in HIVE failing

aliyesami — Thu, 05 May 2016 22:02:30 GMT

I can see the table definitions so it must be able to make database connection ,but insert hangs