Thanks for checking this out! ... View more

core-site has worked for me ... View more

i got the similar issues above with HDP2.6 when i run yum -y erase hdp-select on each host, still there exist problem where Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf User Group mapping (user_group) is missing in the hostLevelParams Kindly advise. ... View more

now, i've solved this issue by setting below property. Now, all mapper/reducer's output will not be stored in memory. But, i need to revisit my table data and predicates(where clause) once again to check if any unnecessary data is fetched. set hive.auto.convert.join=false; ... View more

Falcon supports mirroring for HDFS and Hive. Performance issue I mentioned above is only for HDFS mirroring, if replicated data is not evicted. This is because for Hive mirroring , last successfully replicated event id will be saved in the data store by Falcon and next replication job will start replication past the last successfully replicated event id. Also Falcon cleans up staging paths used for export after the job runs. As DistCP will get only the new data to be replicated there is no performance overhead for Hive mirroring. Just an FYI. ... View more