Member since
11-11-2019
577
Posts
28
Kudos Received
25
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1010 | 02-28-2023 09:32 PM | |
1332 | 02-27-2023 03:33 AM | |
23768 | 12-24-2022 05:56 AM | |
1237 | 12-05-2022 06:17 AM | |
3102 | 11-25-2022 07:37 AM |
12-04-2022
11:04 PM
@Mars001 I see you are using EXTERNAL jdbc driver. COuld you please try with beeline and provide below if this fails: 1. query and queryID 2. hiveserver2 logs 3. Output of show create table <tablename>;
... View more
11-25-2022
07:37 AM
Please find the difference between hive.tez.container.size and tez.task.resource.mb hive.tez.container.size This property specifies tez container size. Usually value of this property should be the same as or a small multiple (1 or 2 times that) of YARN container size yarn.scheduler.minimum-allocation-mb and should not exceed value of yarn.scheduler.maximum-allocation-mb. As a general rule don't put value higher than memory per processor as you want 1 processor per container and you want to spun up multiple containers. You can find very detailed answer and a great architecture diagram on Hortonworks community answer here tez.task.resource.memory.mb Amount of memory used by launched task in TEZ container. tez.task.resource.memory.mb should be set < hive.tez.container.size This will be recalculated. Run the job without setting.
... View more
11-25-2022
02:03 AM
hi @pankshiv1809 DOnt include set tez.task.resource.memory.mb=10240;
... View more
11-24-2022
11:59 PM
@pankshiv1809 Please increase the container size: set hive.tez.container.size=10240; set tez.runtime.io.sort.mb=4096; ==> 40% of hive.tez.container.size Keep on increasing the container. Please also collect table and column stats too https://docs.cloudera.com/HDPDocuments/HDP2/HDP-2.6.5/bk_cloud-data-access/content/hive-analyzing-tables.html Please mark it "Accept As Solution". if your query is answered.
... View more
11-24-2022
12:46 AM
@drgenious Can you please provide below: 1. oozie wf logs(oozie application logs) ==> i can find the exact hive query executed. 2. Hive application logs. Please collect it using: yarn logs -applicationId <appid> > <appid>.txt ==> FOr bith Oozie and hive
... View more
11-23-2022
01:00 AM
@drgenious 1. Impala is always faster. Impala does not use yarn. Impala stores catalog data locally which fetches information faster. Impala backend gthread is built on C++ which is very fast. 2. Impala is not fault tolerant , it is best suited for adhoc queries and ETL is best suited for Hive as Hive is fault tolerant. If the query fails due to network/disk failure,hive will retry but Impala would fail. 3. For stemaming/ingestion like Kafka flow you need to put it in EXTERNAL tables not in Managed(ACID) tables. Managed tabled can be used,if you want to perform alteration of the data like Update/Delete . Please let me know,if you have any queries. Please click "Accept As Solution" , if your query is answered.
... View more
11-22-2022
01:46 AM
Hi @Raed_Faly Could you please share the DDL's for the tables, based on that I can update.
... View more
11-19-2022
12:47 AM
hi @Raed_Faly You need discipline with > 10 gold medals Please use below query and check: select distinct t.discipline, m.team from teams t join medals m on (t.noc=m.team and m.numbergold>10) order by m.team asc limit 5; select distinct t.discipline, m.team from teams t join medals m on (t.noc=m.team and m.numbergold>10) order by m.team desc limit 5; Please "Accept As Solution", if your queries are answered.
... View more
11-19-2022
12:15 AM
@NikoBelic Thanks for the update. The hive metadata indeed is stored on database. You need to take the dump to backup the metadata. Please let me know,if you have nay queries. Please "Accept as Solution" ,if your queries are answered.
... View more
11-16-2022
07:22 PM
@NikoBelic VIEWS are stot=red in TBLS table i.e in Hive metadata. You can create a database dump as a backup. Please follow https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html to create database dump for mysql Please follow https://www.postgresql.org/docs/current/backup-dump.html to create database dump for postgres Please let me know,if you have any queries. Please "Accept As Solution", if your queries are answered.
... View more