Member since
11-02-2017
51
Posts
6
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1113 | 09-22-2022 04:16 AM | |
3508 | 03-09-2018 02:34 PM | |
15971 | 02-01-2018 06:15 AM | |
5718 | 11-13-2017 12:34 PM |
03-06-2018
09:13 AM
@tomoya yoshida It seems you have increased hive.tez.container.size but yarn.scheduler.maximum-allocation-mb is still set to 2250 MB. Can you try increasing yarn.scheduler.maximum-allocation-mb to more than 3750 MB. you need to accordingly set the value of this yarn.nodemanager.resource.memory-mb property. Let me know if it doesn't work after changing this.
... View more
02-01-2018
03:10 PM
Cool ! Please accept the answer. You can execute below command before your select query to get column headers. <code>set hive.cli.print.header=true;
... View more
02-01-2018
06:15 AM
@Carlton Patterson
There is an extra semicolon before TBLPROPERTIES , removing it will solve your problem. Use below script: DROP TABLE IF EXISTS HiveSampleIn;
CREATE EXTERNAL TABLE HiveSampleIn
(
anonid int,
eprofileclass int,
fueltypes STRING,
acorn_category int,
acorn_group STRING,
acorn_type int,
nuts4 STRING,
lacode STRING,
nuts1 STRING,
gspgroup STRING,
ldz STRING,
gas_elec STRING,
gas_tout STRING
) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '10' STORED AS TEXTFILE LOCATION 'wasb://adfgetstarted@geogstoreacct.blob.core.windows.net/samplein/'
TBLPROPERTIES ("skip.header.line.count" = "1"); DROP TABLE IF EXISTS HiveSampleOut;
CREATE EXTERNAL TABLE HiveSampleOut
(
acorn_category int,
acorn_categorycount int )
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '10' STORED AS TEXTFILE LOCATION 'wasb://adfgetstarted@geogstoreacct.blob.core.windows.net/sampleout/'; INSERT OVERWRITE TABLE HiveSampleOut
Select
acorn_category,
count(*) as acorn_categorycount
FROM HiveSampleIn Group by acorn_category;
... View more
01-31-2018
08:09 AM
Create your table as : CREATE EXTERNAL TABLE HiveSampleOut ( acorn_category int, acorn_categorycount int ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '10' STORED AS TEXTFILE LOCATION 'wasb://adfgetstarted@geogstoreacct.blob.core.windows.net/sampleout/'; Or directly create table on the fly while inserting data like: Create table HiveSampleOut As Select acorn_category, count(*)as acorn_categorycount FROM HiveSampleInGroupby acorn_category ;
... View more
01-31-2018
04:19 AM
@Sam Cse
There is an already existing partition at this location hdfs://....../table_name_1/part_col_1=1 or may be a sub-directory and before loading the new partition, cleanup of destination directory is failing. Can you try deleting those files manually and then try Insert Overwrite. Also, share the listing for this hdfs path.
... View more
01-30-2018
07:43 AM
@Hanu V Can you attach a desc extended for your table. Greatest should work provided all the columns have the same datatype and are only primitive types.
... View more
01-30-2018
06:53 AM
@Carlton Patterson
HiveSampleOut has only 1 field acorn_category and you are trying to insert 2 fields into this table acorn_category and acorn_categorycount. Hence, the error. Define the table with 2 fields .
... View more
12-06-2017
08:26 AM
@Devaraj Thaha Great to know that your problem is resolved . Can you please accept this answer . Thanks !
... View more
11-22-2017
02:23 PM
@Saikrishna Tarapareddy Is it about finding missing partitions in Hive Metastore or in HDFS directories ? You can execute " msck repair table <table_name> " command to find out missing partition in Hive Metastore and it will also add partitions if underlying HDFS directories are present. But it will not delete partitions from hive Metastore if underlying HDFS directories are not present . hive> msck repair table mytable;
OK
Partitions missing from filesystem:
... View more
11-13-2017
12:34 PM
2 Kudos
@Devaraj Thaha This is the best example for connecting to hive server from Java using JDBC : https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients#HiveServer2Clients-JDBCClientSampleCode You only need to modify the code little bit to read queries from your file and store the result set to a file instead of printing . But if you are interested in directly executing "hive -f test.hql" from Java then you can create a runtime process and execute it . p = Runtime.getRuntime().exec((new
String[]{"hive","-f","/hive/scripts/test.hql"}));
p.waitFor();
BufferedReader reader =
new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "";
while ((line = reader.readLine())!= null) {
sb.append(line + "\n");
}
... View more
- « Previous
-
- 1
- 2
- Next »