Reply
Highlighted
Explorer
Posts: 25
Registered: ‎02-26-2017
Accepted Solution

How to limit speed ?

Hi , 

 

I want to limit speed of data transformation with CDM because other process*(Not bigdata , not CDH) is using same internet resource with it(CDM). and the process* can't be run as no resource. 

 

How can i set it in CHM?

I tried to set 'DataNode Balancing Bandwidth' in HDFS configuration to 1Mb.

But it looks not working.

 

 

Expert Contributor
Posts: 219
Registered: ‎01-25-2017

Re: How to limit speed ?

Hi @wenjie,

 

Try this:

Maximum Number of Transfer Threads
 
Explorer
Posts: 25
Registered: ‎02-26-2017

Re: How to limit speed ?

Thanks Fawze.

I will try it and take a feedback.
Explorer
Posts: 25
Registered: ‎02-26-2017

Re: How to limit speed ?

It looks it(Maximum Number of Transfer Threads) is not working.

 

I set it from 4096 to 2048 , but NetworkIO still take a high value.

Expert Contributor
Posts: 219
Registered: ‎01-25-2017

Re: How to limit speed ?

The Network IO at the same level when it was 4096?

Explorer
Posts: 25
Registered: ‎02-26-2017

Re: How to limit speed ?

Yes, I set it from 4096 to 2048.
The Network IO at the same level when it was 4096.

Maybe i need to set less number than 2048?
Expert Contributor
Posts: 219
Registered: ‎01-25-2017

Re: How to limit speed ?

Try to do it very low and then check.
Explorer
Posts: 25
Registered: ‎02-26-2017

Re: How to limit speed ?

Sorry for late reply.

I need to test it after business time.

 

I set it to 2046->1024->512-> 256 and found NetWork IO take a low value with the number reducing.

I'm sure the parameter is working for it.

 

For someone who will refer this ticket.

The speed of data transformation will take a very low speed for NetworkIO if you set a low value to this parameter.

so please adjust schedul of jobs.

Announcements