@mbigelow  it come be a dumb question please bear with me .  is it mandatory to use  fully qualified domain name ? because we dont not mention them in any of the hadoop configuration files. when it comes to networking not really my area. could you clarify  ... View more

if it is Vm as you wish since it is a static IP  ... View more

if you are plaining for Single node cluster , then just the hostname would be of your choice .    /etc/hosts 192.168.200.201 Hello /etc/sysconfig/network Hello You have to configure Yum repository , MR2 yarn setup - the below is good link from Cloudera for Single node cluster setup .    https://www.cloudera.com/documentation/enterprise/5-4-x/topics/cdh_qs_yarn_pseudo.html     ... View more

I assume you are using vm .  so edit the file accordingly , to perform the SSH configuration follow the link    On both the nodes .    /etc/hosts 192.168.200.201 master 192.168.200.202 slave On the master node /etc/sysconfig/network master on the slave node   /etc/sysconfig/network slave Below is the best link to walk through the rest of the configuration ,mate    https://www.tutorialspoint.com/hadoop/hadoop_multi_node_cluster.htm ... View more

Gotcha . its just matter of two files that you need to tweak. are you plaining to perform single node or multi node cluster ?  ... View more

were you able to fix it ? if not i can help you  that ... View more

@mbigelow Sorry for the late response , was on Vacation :))  bleow is my answer to the questionaire    is size the amount after the merge? Yes it . What was the average size before? between 50kb to 100 kb . How long did it take to run? 10-15 minutes           ... View more

Because you defined a new the table from outside of impala , you need to refresh hive metadata in impala.   fire this command in the impala-shell or Hue INVALIDATE METADATA You should be able to query the table ... View more

@mbigelow    Parquet - Merge   No. of files Ulimit Size (GB) 5000 15000 2.1 4140 10000 1.4 System Configuration   model name : Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz cpu MHz : 1257.457 cache size : 20480 KB cpu cores : 8 processor : 15 cpu MHz : 1212.914 cache size : 20480 KB siblings : 16 core id : 7 cpu cores : 8   The above error got fixed as per your suggestion by increasing the  Ulimit . But do you think the above benchmark is good or should we decrease the number of files in the  folder . Please let me know your thoughts   Cant thank you enough for the help .   ... View more

@mbigelow   Thanks for the quick trurn around . Will try and let you know the results.      ... View more