@anirbandd  You can write your own custom reducer class things like LazyOutputFormat ,etc   i believe there is no property that you can tweak in your mapred or hive xml for your custom file output format while performing in  hive    ... View more

Check if your ntp is synhronised  ... View more

TO VIEW THE POOL CONFIGURATION   YARN -> CONFIGURATION -> SEARCH screen  YARN - screen - double check  ... View more

Sure thing .    Screen 1 Screen 2 Screen 3 Screen - Yarn Configuration   Let me know if you still have trouble viewing it  ... View more

Just remove the port number , try the below command as a test  .    sqoop list-tables --connect jdbc:mysql://quickstart.cloudera/retail_db --username retail_dba --password clouderaF   I tested it - it worked                For your sqoop command . perform the below mate sqoop import -m 2 --connect jdbc:mysql://quickstart.cloudera/retail_db --username=retail_dba --password=cloudera --table departments --compression-codec=snappy --as-parquetfile --warehouse-dir=/user/hive/warehouse/sriram/ --hive-import  Let me know if that works .  ... View more

Could you please refer the below link , will give you some insight in creating the HDFS directory structure (best practices) https://www.quora.com/What-is-the-best-directory-structure-to-store-different-types-of-logs-in-HDFS-over-the-time/answer/Eric-Sammer   Also it is always recommended to have odd numbers when you configure Mater nodes. As you will be using HA  / Zookeeper .  More of like 3 Master .    You can create database in Hive and create tables underneath as you do in Oracle ,Create / Give permission to the schema Owner. You dont need a seperate Hive Metastore. You can also configure a Shared Metastore where Both Hive / Impala can use em.    ... View more

You can have seprate database on Hive Metastore  for all the three enviroments ,plus the HDFS directory structure I would strongly recommend you to have sperate Hadoop clusters for your production . Because you will have 2 or more Hadoop Namenode HA or Resourcemanager HA configured for all the three enviroments maintaining them  will be a cubersome if there is going to be heavy load on all the three enviroment on a single Hadoop Clusters   What you can do is isolate the production from the other two enviroment have a sperate Hadoop clusters .   And I assume you are plaining to  have a shared Metastore for HIVE / IMPALA .  You should also take in to consideration of other eco systems .  Please let me know if you need more information.  ... View more

This is wrong  [localhost.localdomain:21000] > create table emp2 LIKE select dept,sal,loc from emp; Query: create table emp2 LIKE select dept,sal,loc from emp ERROR: AnalysisException: Syntax error at: create table emp2 LIKE select dept,sal,loc from emp This is correct syntax  create table emp2 LIKE emp; Try this  create table emp2 AS select dept,sal,loc from svk.emp; The wired part is I am able to perform the same syntax with "as"  as small refer the screen  You have got some formatting issue check that your query does not have any unwanted space and mostly like just type it instead of copy and paste .    ... View more