Hi Sam2020,   Just wanna add,I encounter this issue as well. Got the exact same error while downloading the parcel from Cloudera Manager. Im using local/internal repository.    My permission on /opt/cloudera/parcel-repo is already cloudera-scm without any modification needed. And the sha file also does not have 1 or 256 at the end.   What I did was: 1.Remove old unused 7.1.7 p0 parcels. 2.Restart CM server 3.Redownload 7.1.7 p74 from CM UI.   *Im on cdh 6 planning to upgrade to 7.1.7 p74 without doing the 7.1.7 p0 *I do realize that all other parcels including 7.1.7 p0 have .torrent file except this 7.1.7 p74. Not sure if that have anything to do with it ... View more

Hi @GangWar  Thank you very much for your response.  I did suspect something got to do with my java version and I've already did what you mention to disable the referrals setting sun.security.krb5.disableReferrals=true but the zookeeper still unable to start.   On the problematic cluster, I'm using OpenJDK 1.8.0u262. I have one more kerberized cluster that is running fine using OpenJDK 1.8.0u312.So what other things I tried previously.    1. Downgraded my OpenJDK to match the problematic version u262.   2. Restarted cluster few times.   3. Cluster still working fine with Kerberos, no need to comment renew_lifetime That is why I ignored the Java version suspicion.   So the only thing for now that can make my zookeeper start was by commenting on the renew_lifetime. This guy have the same exact thing with my problem and solution. He did try the referrals as well. Do you think there are any other bugs related to this problem?   https://community.cloudera.com/t5/Community-Articles/How-to-solve-the-Message-stream-modified-41-error-on/ta-p/292986   Thank you and regards. Mus         ... View more

Hi, went through my testing again.  Unfortunately, I missed the step where I need to change/add the parameters on my commandline. Change my TeraGen, TeraSort and TeraValidate parameter and got better results   TeraGen: 1 min 57 sec TeraSort: 22min 55sec TeraValidate: 1 min 23sec.   Thank you very much for your writeup again.  ... View more

Hi @sunile_manjee . Thank you very much for this excellent writeup and testing.   I got the chance to run TeraGen, Terasort and TeraValidate with an environment which have around 40+ worker node with 3 master recently. Comparing to your results, it seems that for the Teragen mine is quite bad.    My results: Teragen 57mnt 44sec, TeraSort 49mnt 01sec TeraValidate 4mnt16sec   My spec: Masternode,cpu:16core , RAM:384GB Storage:12x2TB SATA 6Gb 7.2K RPM Workernode,cpu:24core, RAM;384GB Storage: 12x2TB NL SAS  12Gb 7.2k RPM network badwidth:20gbps   My observation It believe the speed of the disk is very important here. Since my disk is only using SATA&SAS instead of SSD, I'm assuming this is the core reason why my Teragen results is bad even I got more nodes. Please correct me if this is not the case. But one thing that I saw from your BigStep test, the workernode is using local HDD not SSD like the AWS nodes. Shouldn't this decrease the TeraGen results? but it was the fastest one at 11min 49seconds.   I also read from your test here https://community.cloudera.com/t5/Community-Articles/More-Hadoop-nodes-faster-IO-and-processing-time/ta-p/247155. The performance is way better using 5nodes against 3nodes. I wass hoping my 40+ node will top this, but it is not. Any advice is very very appreciated.   P/s: I play with YARN configuration to try to get a better result. I set Map&Reduce cores&memory to 8 & 64GB. Do you think this help? Default setting (1core and 1gb ram)will make my terasort crash. ... View more