@Elbehwash    1. Please navigate to CM ==> Hive on tez ==>  COnfiguration ==> Hiveserver2 safety valve for hive-site.xml 2. Add      hive.server2.tez.initialize.default.sessions = false;   3. restart and check   ... View more

Please find the difference between hive.tez.container.size and tez.task.resource.mb   hive.tez.container.size This property specifies tez container size. Usually value of this property should be the same as or a small multiple (1 or 2 times that) of YARN container size yarn.scheduler.minimum-allocation-mb and should not exceed value of yarn.scheduler.maximum-allocation-mb. As a general rule don't put value higher than memory per processor as you want 1 processor per container and you want to spun up multiple containers. You can find very detailed answer and a great architecture diagram on Hortonworks community answer here tez.task.resource.memory.mb Amount of memory used by launched task in TEZ container. tez.task.resource.memory.mb should be set < hive.tez.container.size This will be recalculated. Run the job without setting.   ... View more

@drgenious    1. Impala is always faster. Impala does not use yarn. Impala stores catalog data locally which fetches information faster. Impala backend gthread is built on C++ which is very fast. 2. Impala is not fault tolerant , it is best suited for adhoc queries and ETL is best suited for Hive as Hive is fault tolerant. If the query fails due to network/disk failure,hive will retry but Impala would fail. 3. For stemaming/ingestion like Kafka flow you need to put it in EXTERNAL tables not in Managed(ACID) tables. Managed tabled can be used,if you want to perform alteration of the data like Update/Delete . Please let me know,if you have any queries. Please click "Accept As Solution" , if your query is answered. ... View more