Reply
Contributor
Posts: 27
Registered: ‎10-04-2017

Parcel distribution is very slow

Hi,

 

We faced a very slow parcel distribution while adding new hosts to the cluster. A generic scp was transferring @100Mbps while CM was distributing parcels at few kbps. It took 3 hours to distribute parcels to 10 hosts that were in the same subnet. The network speed in the servers is 20Gbps. There were no exceptions in the agent and server logs. Below are the logs when the CDH parcel distribution finished.

 

 

2017-11-10 18:19:23,821 WARN 1513345101@scm-web-29149:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-11-10 18:19:23,821 WARN 1513345101@scm-web-29149:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-11-10 18:19:23,821 WARN 1513345101@scm-web-29149:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-11-10 18:19:23,821 WARN 1513345101@scm-web-29149:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-11-10 18:19:23,821 WARN 1513345101@scm-web-29149:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-11-10 18:19:23,821 WARN 1513345101@scm-web-29149:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-11-10 18:19:24,220 INFO 1965623353@scm-web-28960:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.5.4-1.cdh5.5.4.p0.9-el6.parcel (1457226645 bytes) to 180.39.3.93 over PT9991.723S. Rate: 0.139 MB/s
2017-11-10 18:19:24,220 INFO 1965623353@scm-web-28960:com.cloudera.server.web.common.RequestTimeInterceptor: Slow request: GET /parcel/download/CDH-5.5.4-1.cdh5.5.4.p0.9-el6.parcel in 9991.72432379 seconds.
2017-11-10 18:19:25,170 INFO 1748351042@scm-web-28971:com.cloudera.server.web.cmf.ParcelController: Served parcel CDH-5.5.4-1.cdh5.5.4.p0.9-el6.parcel (1457226645 bytes) to 180.39.3.89 over PT10000.347S. Rate: 0.139 MB/s
2017-11-10 18:19:25,171 INFO 1748351042@scm-web-28971:com.cloudera.server.web.common.RequestTimeInterceptor: Slow request: GET /parcel/download/CDH-5.5.4-1.cdh5.5.4.p0.9-el6.parcel in 10000.349528177 seconds.
2017-11-10 18:19:25,592 WARN 1513345101@scm-web-29149:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING
2017-11-10 18:19:25,592 WARN 1513345101@scm-web-29149:com.cloudera.parcel.ClusterParcelStatus: Parcel not distributed but have active state ACTIVATING

Announcements