Reply
Highlighted
Contributor
Posts: 78
Registered: ‎06-19-2014

With sentry and ldap,can't load data from hdfs

HI:

 

I had created a new user and group (bi:bi) on the hadoop file system.

hadoop fs -ls /user/bi
drwxr-xr-x   - bi bi          0 2015-03-05 03:10 /user/bi/bi_env
drwxr-xr-x   - bi bi          0 2015-03-05 07:28 /user/bi/tmp

and grant all privileges 

show grant role user_bi_all_role;     
+-------------------------------------------------+--------+------------+---------+-------------------+-----------------+------------+---------------+-------------------+----------+--+
|                    database                     | table  | partition  | column  |  principal_name   | principal_type  | privilege  | grant_option  |    grant_time     | grantor  |
+-------------------------------------------------+--------+------------+---------+-------------------+-----------------+------------+---------------+-------------------+----------+--+
| hdfs://masterhost/user/bi                       |        |            |         | user_bi_all_role  | ROLE            | *          | false         | 1425608038239000  | --       |
| hdfs://masthost:8020/user/bi  |        |            |         | user_bi_all_role  | ROLE            | *          | false         | 1425634980906000  | --       |
| hdfs:///user/bi                                 |        |            |         | user_bi_all_role  | ROLE            | *          | false         | 1425608431493000  | --       |

 

show grant role bi_system_all_role;   
+------------------------------------------------+--------+------------+---------+---------------------+-----------------+------------+---------------+-------------------+----------+--+
|                    database                    | table  | partition  | column  |   principal_name    | principal_type  | privilege  | grant_option  |    grant_time     | grantor  |
+------------------------------------------------+--------+------------+---------+---------------------+-----------------+------------+---------------+-------------------+----------+--+
| bi_system                                      |        |            |         | bi_system_all_role  | ROLE            | *          | false         | 1425476285460000  | --       |

 

But now ,I want to load data from /user/bi uri to a table in the database bi_system(beeline with ldap bi user and password)

0: jdbc:hive2://hiveserverhost:10> LOAD DATA INPATH 'hdfs:///user/bi/ip_info' OVERWRITE INTO TABLE ip_info ;         
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask (state=08S01,code=1)

 the hive server2 log:

2015-03-09 06:14:21,624 INFO org.apache.hadoop.hive.ql.log.PerfLogger: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
2015-03-09 06:14:21,624 INFO ZooKeeperHiveLockManager:  about to release lock for bi_system/ip_info
2015-03-09 06:14:21,640 INFO ZooKeeperHiveLockManager:  about to release lock for bi_system
2015-03-09 06:14:21,646 INFO org.apache.hadoop.hive.ql.log.PerfLogger: </PERFLOG method=releaseLocks start=1425881661624 end=1425881661646 duration=22 from=org.apache.hadoop.hive.ql.Driver>
2015-03-09 06:14:21,646 ERROR org.apache.hive.service.cli.operation.Operation: Error running hive query: 
org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask
        at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:147)
        at org.apache.hive.service.cli.operation.SQLOperation.access$000(SQLOperation.java:69)
        at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:200)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
        at org.apache.hadoop.hive.shims.HadoopShimsSecure.doAs(HadoopShimsSecure.java:502)
        at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:213)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

  How to fix it?

 

  Thanks very much!

 

 

rube