Thanks, What I am experiencing is that the complete file, if 300GB, has to be assembled before upload to S3. This requires either 300GB of memory or disk. Distcp does not create a part file per block. I have not witnessed any file split being done. Multi part uploads require you get an upload ID and upload many part files with a numeric extension and in the end ask S3 to put them back together. I do not see any of this being done. I admit I do not know much about all this and it could be happening out of my sight. ... View more

@Surya Rao Although It is strongly recommended to not change hostname/IPAddress of HDP cluster nodes. Hortonworks support / engineering assistance might be needed in such scenario as it might impact the whole cluster. . You also might want to refer to the following article: https://community.hortonworks.com/content/supportkb/48815/how-to-change-ip-address-for-a-datanode.html ... View more

@Deepak Sharma, Thank you for the quick reply.. I was trying to bring the github code to easily readable format in excel spreadsheet. For instance, for HDFS service the service checks are as follows.. Trying to prepare the same for rest of the services.. if anyone already did this exercise, please save my time. 🙂 Thanks Surya ... View more