- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Cannot select * from tables in hive
- Labels:
-
Apache Hive
Created on ‎12-16-2016 03:23 PM - edited ‎09-16-2022 03:51 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
After upgrading to a new version of CM, I am unable to list my tables in hive.
select * from prod_83.std_clnt_sku limit 5;
Error: Error while compiling statement: FAILED: SemanticException Unable to determine if hdfs://nameservice1/user/hive/warehouse/prod_83.db/std_clnt_sku is encrypted: java.lang.IllegalArgumentException: Wrong FS: hdfs://nameservice1/user/hive/warehouse/prod_83.db/std_clnt_sku, expected: hdfs://zeus.corpdom.com:8020 (state=42000,code=40000)
The Location of the database is listed as:
hdfs://nameservice1/user/hive/warehouse/prod_83.db/std_clnt_sku
I can see it inside hdfs as
drwxrwx--x+ - analytics sentry_rwx 0 2016-10-17 09:49 /user/hive/warehouse/prod_83.db/std_clnt_sku
Can you help me solve this problem?
Thank You
Jamie
Created ‎12-20-2016 09:54 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Did you enable High Availability by any chance?
Did you chance your dfs.nameservices by any chance?
Created ‎12-17-2016 03:52 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@cpuengr Pls check with your admin that any security implemented as part of upgrade. I can see the keyword sentry in your hdfs path. so you might need a proper role level access to query your table. Also + denotes ACL implemention. Are you using same login for your hive and hdfs? In case if you are using Hue for Hive
You will get very highlevel information about security in this path
https://community.cloudera.com/t5/Security-Apache-Sentry/Hadoop-Security-for-beginners/m-p/48576
Thanks
Kumar
Created ‎12-20-2016 09:02 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It looks to me like all security is fine and I should have access to these databases.
hdfs dfs -getfacl /user/hive/warehouse
# file: /user/hive/warehouse
# owner: hive
# group: sentry_rwx
user::rwx
group::rwx
group:admin:rwx
group:sentry_rwx:rwx
id uname
uid=1009(uname) gid=1011(uname) groups=1006(admin),1011(uname)
Is there somewhere else I need to look?
Thank You
Created ‎12-20-2016 09:40 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Did you rename your cluster as part of upgrade? By default the name is nameservice1. Did you modify it as zeus.corpdom.com by any chance?
Login to CM -> Left Top green button name
Also check your table location from the below command.
describe formatted tablename;
Thanks
Kumar
Created ‎12-20-2016 09:45 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The name of my cluster has always been gods. I did not change it, and it is not nameservice1.
Created ‎12-20-2016 09:54 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Did you enable High Availability by any chance?
Did you chance your dfs.nameservices by any chance?
Created ‎12-20-2016 10:10 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
To make it more easier
CM -> HDFS -> Configuration -> NameNode Nameservice
I assume it was "nameservice1" before and "zeus.corpdom.com now".
Also did you check the location from "describe formatted tablename" in Hive. If the location refers to nameservice1 then change either NameNode Nameservice name or Hive table location. If you have very less tables, then I would recommand to change the hive table location to match with NameService.
Hope this helps!
Created ‎12-20-2016 10:11 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Before the upgrade the cluster was HA, but after it will not let me set up the HA anymore. It broke it apart, and I am trying to figure that out as well.
I looked at the name of the dfs.nameservices and they were empty so I now have nameservice1 in there and it still is not working.
Thank you
Jamie
Created ‎12-20-2016 10:12 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I still didn't get answer for describe formatted tablename!!
Created ‎12-20-2016 10:15 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I posted that above in another comment. The database is listed here..
hdfs://nameservice1/user/hive/warehouse/prod_83.db/std_clnt_sku
