Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

with sentry and ldap,hive can't load data from hdfs or local file

Highlighted

with sentry and ldap,hive can't load data from hdfs or local file

Contributor

HI:

 

I had created a new user and group (bi:bi) on the hadoop file system.

hadoop fs -ls /user/bi
drwxr-xr-x   - bi bi          0 2015-03-05 03:10 /user/bi/bi_env
drwxr-xr-x   - bi bi          0 2015-03-05 07:28 /user/bi/tmp

and grant all privileges 

show grant role user_bi_all_role;     
+-------------------------------------------------+--------+------------+---------+-------------------+-----------------+------------+---------------+-------------------+----------+--+
|                    database                     | table  | partition  | column  |  principal_name   | principal_type  | privilege  | grant_option  |    grant_time     | grantor  |
+-------------------------------------------------+--------+------------+---------+-------------------+-----------------+------------+---------------+-------------------+----------+--+
| hdfs://masterhost/user/bi                       |        |            |         | user_bi_all_role  | ROLE            | *          | false         | 1425608038239000  | --       |
| hdfs://masthost:8020/user/bi  |        |            |         | user_bi_all_role  | ROLE            | *          | false         | 1425634980906000  | --       |
| hdfs:///user/bi                                 |        |            |         | user_bi_all_role  | ROLE            | *          | false         | 1425608431493000  | --       |

 

show grant role bi_system_all_role;   
+------------------------------------------------+--------+------------+---------+---------------------+-----------------+------------+---------------+-------------------+----------+--+
|                    database                    | table  | partition  | column  |   principal_name    | principal_type  | privilege  | grant_option  |    grant_time     | grantor  |
+------------------------------------------------+--------+------------+---------+---------------------+-----------------+------------+---------------+-------------------+----------+--+
| bi_system                                      |        |            |         | bi_system_all_role  | ROLE            | *          | false         | 1425476285460000  | --       |

 

But now ,I want to load data from /user/bi uri to a table in the database bi_system(beeline with ldap bi user and password)

0: jdbc:hive2://hiveserverhost:10> LOAD DATA INPATH 'hdfs:///user/bi/ip_info' OVERWRITE INTO TABLE ip_info ;         
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask (state=08S01,code=1)

 the hive server2 log:

2015-03-09 06:14:21,624 INFO org.apache.hadoop.hive.ql.log.PerfLogger: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
2015-03-09 06:14:21,624 INFO ZooKeeperHiveLockManager:  about to release lock for bi_system/ip_info
2015-03-09 06:14:21,640 INFO ZooKeeperHiveLockManager:  about to release lock for bi_system
2015-03-09 06:14:21,646 INFO org.apache.hadoop.hive.ql.log.PerfLogger: </PERFLOG method=releaseLocks start=1425881661624 end=1425881661646 duration=22 from=org.apache.hadoop.hive.ql.Driver>
2015-03-09 06:14:21,646 ERROR org.apache.hive.service.cli.operation.Operation: Error running hive query: 
org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
        at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:147)
        at org.apache.hive.service.cli.operation.SQLOperation.access$000(SQLOperation.java:69)
        at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:200)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
        at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:502)
        at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:213)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

if I load data from local file(fie /home/bi/aa.txt exist.)

0: jdbc:hive2://hiveserver2host:10> LOAD DATA  INPATH '/home/bi/aa.txt' OVERWRITE INTO TABLE test_qiu ;        
Error: Error while compiling statement: FAILED: SemanticException Line 1:18 Invalid path ''file:///home/bi/aa.txt'': No files matching path file:/home/bi/aa.txt (state=42000,code=40000)

 

How to fix it?

 

  Thanks very much!

 

 

rube

1 REPLY 1

Re: with sentry and ldap,hive can't load data from hdfs or local file

Contributor
I fix it with acl,the command like that : hadoop fs -setfacl -R -m user:hive:rwx /user/bi Is that the right way? Where is the cloduera enginer....