Member since
03-07-2019
322
Posts
17
Kudos Received
9
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
330 | 07-31-2024 11:57 AM | |
1666 | 03-27-2019 04:52 AM | |
6398 | 11-21-2018 10:21 PM | |
12801 | 09-14-2016 07:35 PM | |
10866 | 07-01-2016 06:56 PM |
07-31-2024
11:57 AM
This is currently not supported. I would recommend you to raise a git issue (enhancement or feature) for this and add in the usecase and other details there so the team would review and take it forward. https://github.com/cloudera/hue/issues
... View more
04-27-2022
09:37 AM
The root cause of this is PEP 3151, introduced in Python 3.3: PEP 3151 – Reworking the OS and IO exception hierarchy Python 3.3 release notes You can overcome this issue with the following changes in the file /usr/lib64/python2.7/test/test_support.py From: def _is_ipv6_enabled():
"""Check whether IPv6 is enabled on this host."""
if socket.has_ipv6:
sock = None
try:
sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
sock.bind((HOSTv6, 0))
return True
except OSError:
pass
finally:
if sock:
sock.close()
return False To: def _is_ipv6_enabled():
"""Check whether IPv6 is enabled on this host."""
if socket.has_ipv6:
sock = None
try:
sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
sock.bind((HOSTv6, 0))
return True
except socket.error if sys.version_info < (3, 3) else OSError: ---> this is how it should be
pass
finally:
if sock:
sock.close()
... View more
03-27-2019
04:52 AM
1 Kudo
@Artur Brandys, Just saw this question not sure if you have already found the answer or not. No, you cannot have a different database for users as these are internal to Hue's django and how it works with the tables by taking the information from Hue.ini.
... View more
11-22-2018
11:07 AM
I created /etc/hive/conf/beeline-hs2-connection.xml and it worked. Thanks
... View more
11-23-2018
11:18 AM
@Prabhu M @Mahesh Balakrishnan : Issue has been resolved. It's a bug with Tez version 0.9.0 (HDP-3.0.0.0) which they have fixed in Tez 0.9.1 version(HDP-3.0.1.0)
... View more
03-04-2019
07:51 AM
you can modify hive.distro script and let the login authentication enter in the script it self
... View more
07-05-2016
05:32 PM
1 Kudo
Hi @Johnny Fuger. When you have a set of files in an existing directory structure, and you are not able to move the files around, there is a way to create a Hive table that is partitioned. You can manually define the partitions (explicitly). It is important to note that you are controlling each partition. You create the table, then add each partition manually via an ALTER TABLE command. Here is an example where there are 3 days worth of files in three different directories: directory #1 has 1 file (10 records total), the second directory has 2 files(20 records total), and the 3rd has 3 files(30 records total): hadoop fs -mkdir -p /user/test/data/2016-07-01
hadoop fs -mkdir -p /user/test/data/2016-07-02
hadoop fs -mkdir -p /user/test/data/2016-07-03
hadoop fs -put /tmp/poc_data_file.txt /user/test/data/2016-07-01
hadoop fs -put /tmp/poc_data_file.txt /user/test/data/2016-07-02/poc_data_file2.txt
hadoop fs -put /tmp/poc_data_file.txt /user/test/data/2016-07-02
hadoop fs -put /tmp/poc_data_file.txt /user/test/data/2016-07-03
hadoop fs -put /tmp/poc_data_file.txt /user/test/data/2016-07-03/poc_data_file2.txt
hadoop fs -put /tmp/poc_data_file.txt /user/test/data/2016-07-03/poc_data_file3.txt
[root@sandbox hdfs]# hadoop fs -ls -R /user/test/data
drwxr-xr-x - hdfs hdfs 0 2016-07-01 22:30 /user/test/data/2016-07-01
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:30 /user/test/data/2016-07-01/poc_data_file.txt
drwxr-xr-x - hdfs hdfs 0 2016-07-01 22:32 /user/test/data/2016-07-02
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:32 /user/test/data/2016-07-02/poc_data_file.txt
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:31 /user/test/data/2016-07-02/poc_data_file2.txt
drwxr-xr-x - hdfs hdfs 0 2016-07-01 22:32 /user/test/data/2016-07-03
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:32 /user/test/data/2016-07-03/poc_data_file.txt
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:32 /user/test/data/2016-07-03/poc_data_file2.txt
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:32 /user/test/data/2016-07-03/poc_data_file3.txt
Now create an external table with a partition clause. Note the rowcount is zero initially since we have not defined any partitions yet. create external table file_data_partitioned (id int, textval string, amount double)
partitioned by (dateval string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
LOCATION '/user/test/data';
select count(*) from file_data_partitioned;
hive> select count(*) from file_data_partitioned;
OK
0 Now manually define the 3 partitions on the data using ALTER TABLE commands. You need to specify the correct location for each partition. These partitions could be anywhere in HDFS. -----------------------------------------------
-- Add partitions manually
-----------------------------------------------
alter table file_data_partitioned add partition (dateval = '2016-07-01')
location '/user/test/data/2016-07-01';
alter table file_data_partitioned add partition (dateval = '2016-07-02')
location '/user/test/data/2016-07-02';
alter table file_data_partitioned add partition (dateval = '2016-07-03')
location '/user/test/data/2016-07-03';
---------------------------------------
-- Run statistics
---------------------------------------
analyze table file_data_partitioned compute statistics ; Now we can see & query the data in each partition. hive> select dateval, count(*)
> from file_data_partitioned
> group by dateval;
OK
2016-07-01 10
2016-07-02 20
2016-07-03 30 Important note though - if you choose this method of manual partitioning, you should always do it the same way each time you add data to the table. Otherwise you will get different directory structures in HDFS for the same table - data will be spread out among the cluster, which can get messy. Here's an example of this when you do an INSERT INTO command to create data for Partition 2017-07-31: insert into file_data_partitioned partition (dateval = '2016-07-31')
select id, textval, amount
from file_data_partitioned
where dateval = '2016-07-01';
[root@sandbox hdfs]# hadoop fs -ls -R /user/test/data
drwxr-xr-x - hdfs hdfs 0 2016-07-01 22:30 /user/test/data/2016-07-01
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:30 /user/test/data/2016-07-01/poc_data_file.txt
drwxr-xr-x - hdfs hdfs 0 2016-07-01 22:32 /user/test/data/2016-07-02
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:32 /user/test/data/2016-07-02/poc_data_file.txt
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:31 /user/test/data/2016-07-02/poc_data_file2.txt
drwxr-xr-x - hdfs hdfs 0 2016-07-01 22:32 /user/test/data/2016-07-03
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:32 /user/test/data/2016-07-03/poc_data_file.txt
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:32 /user/test/data/2016-07-03/poc_data_file2.txt
-rw-r--r-- 1 hdfs hdfs 1024 2016-07-01 22:32 /user/test/data/2016-07-03/poc_data_file3.txt
drwxr-xr-x - hdfs hdfs 0 2016-07-05 16:53 /user/test/data/dateval=2016-07-31
-rwxr-xr-x 1 hdfs hdfs 182 2016-07-05 16:53 /user/test/data/dateval=2016-07-31/000000_0
Note the new directory created for 2016-07-31 and see that it has a different structure - the default structure that Hive uses when Hive controls partitioning ( ... /dateval=2016-07-31/ ...) I hope this helps.
... View more
07-18-2018
07:40 AM
synchronize the Tez configurations on all nodes, and restart hiveserver2, it should work fine.
... View more