Hi i have a small application that generates some reports without using any map reduce code i want to understand what are the real benefits of using Data lake, i think it will be useful for enterprise if there are many products which are writing data to various hadoop clusters and in order to have unified view of the various issues and having common data store , apart from this what are the other real benefits ? How does data lake work if i want particular HDP version ? i think its easier to switch to particular HDP in a separate cluster from ambari but what about data lake? also if multiple applications use data lake and just 1 application require frequent changes like hbase coprocessor for testing various things , is it advisable to go for data lake ? HA we get in cluster as well , so what are the main advantages technically if we dont bother cost ... View more

Hi Team, is anyone aware of issues during installation , that why do we get so many broken symlink issues during installation , this issue i faced in HDP 2.3.4 and ambari-2.2.2.0 Please see below https://community.hortonworks.com/questions/33492/hdp-234-failed-parent-directory-usrhdpcurrenthadoo.html i was installing 3 Node HDP 2.4.0.0 cluster, wherein at step "Install,Start and Test" on 1 node installation went fine but on other 2 nodes there were random symlink issues i had to fix broken symlink issues manually most of the times and final after spending so much time i was able to successfully installed HDP 2.4.0.0 issues like as below and shown in image 2016-08-18 21:20:17,474 - Directory['/etc/hive'] {'mode': 0755} 2016-08-18 21:20:17,474 - Directory['/usr/hdp/current/hive-client/conf'] {'owner': 'hive', 'group': 'hadoop', 'recursive': True} 2016-08-18 21:20:17,474 - Creating directory Directory['/usr/hdp/current/hive-client/conf'] since it doesn't exist I had proper prerequisites available before starting installation as given in http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.2.0/bk_Installing_HDP_AMB/content/_hdp_24_repositories.html also randomly doing retries it works 😞 Please advise if you think i am doing something wrong or any good best practices for installation and debugging Thanks, ... View more

Hi team, i see HDP stores its lib files and packages in /usr/hdp and maintains diff versions can we control HDP installation packages or rpm and make installation relocatable to other directories like /opt if my It team does not permit installation inside /usr then what to do ? # ls /usr/hdp/ 2.4.0.0-169 2.4.2.0-258 current Please advise rpm -ql hdp-select-2.4.2.0-258.el6.noarch /usr/bin/conf-select /usr/bin/hdp-select ... View more

Hi Team, we see logs of various hadoop services are stored in /var/log can we change it to our customized location if i dont want to store logs in below location then /var/log/ambari-agent/ /var/log/ambari-metrics-monitor/ /var/log/ambari-server/ /var/log/hbase /var/log/zookeeper i see in ambari changing log location is disabled ? ... View more

Hi Team, My hadoop namenode servers are without HBA and but servers are RAID 10 so do i need NFS point to save namenode metada file edits etc on NFS location as well if i have active namenode as well in cluster also my question is if my hardware is without HBA storage and RAID 10 so can i connect to NFS point from such hardware ? basically what are the recommendations for namenode HA ? ... View more