Member since
09-25-2015
356
Posts
382
Kudos Received
62
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2438 | 11-03-2017 09:16 PM | |
1915 | 10-17-2017 09:48 PM | |
3802 | 09-18-2017 08:33 PM | |
4507 | 08-04-2017 04:14 PM | |
3456 | 05-19-2017 06:53 AM |
06-09-2017
04:38 PM
1 Kudo
There are two problems here with the DDL: Use of delimiter with ORC. ORC is in itself a format so you don't have to provide a delimiter. You would define an external table with a location, also ACID tables cannot be external tables, see here. The proper definition for this table would be: CREATE TABLE FIREWALL_LOGS(
time STRING,
ip STRING,
country STRING,
status INT
)
CLUSTERED BY (time) into 25 buckets
STORED AS ORC
TBLPROPERTIES("transactional"="true");
... View more
05-26-2017
08:31 PM
2 Kudos
Hive uses the MySQL jdbc connector for issuing queries to MySQL. You can turn on logging through the jdbc connection url (append &profileSQL=true&logger=com.mysql.jdbc.log.StandardLogger) you are using to connect to mysql db in hive-site.xml. To turn it on the MySQL server you need to set the following in the my.cnf. general_log = 1
general_log_file = /path/to/query.log
See more information here.
... View more
05-24-2017
03:27 PM
desc <tablename> shows the table definition (including column names with the datatype) while select * from <tablename> shows the data in the table.
... View more
05-22-2017
05:03 PM
1 Kudo
Did you verify the data by viewing this data using beeline on the cluster? Does it show up correctly lining up under the expected columns when you do a select * from <tablename>. You can run desc [extended|formatted] <tablename> to view the metadata for the table.
... View more
05-19-2017
06:53 AM
1 Kudo
I suspect you are hitting a Postgres bug, see https://github.com/pgjdbc/pgjdbc/issues/667. Can you try changing the Postgres JDBC driver to a newer version?
... View more
05-18-2017
07:57 PM
1 Kudo
Overall information is available across partitions only if you are on HDP-2.6 and using Hive2 which i see you are currently not from your description (1.2.1.2.5). If you are using hive 2 the information would be something like: beeline> desc formatted `test_table`;
| Table Parameters: | | |
| | COLUMN_STATS_ACCURATE | {\"BASIC_STATS\":\"true\"} |
| | numFiles | 3 |
| | numRows | 5 |
| | rawDataSize | 21 |
| | totalSize | 26 |
| | transient_lastDdlTime | 1495086467 | In case of Hive 1 unfortunately you will have to run the command per partition and aggregate manually.
... View more
05-18-2017
06:07 AM
1 Kudo
The stats for partitioned table are available per partition, you can do desc formatted, example: hive> desc formatted `test_table` partition(`date`='2016-12-30');
...
Partition Parameters:
COLUMN_STATS_ACCURATE {\"BASIC_STATS\":\"true\"}
numFiles 1
numRows 2
rawDataSize 9
totalSize 11
transient_lastDdlTime 1495086721
... View more
05-12-2017
02:26 AM
1 Kudo
For enabling the SQL Std Auth manually you would need to set the following settings in the hive-site.xml before restarting the HiveServer2. hive.server2.enable.doAs=false
hive.security.authorization.enabled=true
hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory
hive.security.authenticator.manager=org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator
hive.users.in.admin.role=<admin user>
In the above <admin user> would be replaced by the user you want as admin. Note that you can perform admin operations after you connect to your beeline session as the admin user and then run the following command: set role ADMIN; For an Ambari controlled cluster recommendation would be to enable it through toggle on the Hive configs page.
... View more
05-03-2017
04:06 AM
1 Kudo
Can you try adding the property hadoop.security.auth_to_local to hive-site.xml as well, bounce the hive services and then try?
... View more
04-28-2017
09:40 PM
2 Kudos
The official hue rpm that is available with HDP-2.5/2.6 is hue-2.6.1. I don't see anyway to get more recent hue rpms for HDP stack.
... View more