question Re: Why it costs about 2 hours when we start nifi with 2000 process groups. in Archives of Support Questions (Read Only)

Why it costs about 2 hours when we start nifi with 2000 process groups.

hailong_yang — Wed, 31 Jan 2018 15:03:20 GMT

Our project use kylo which based on nifi to ETL data from database to hive,but we find that nifi start more and more slow with the feed in kylo more and more . Now it costs 2 hours which kylo has 2000 feeds.

Re: Why it costs about 2 hours when we start nifi with 2000 process groups.

dchaffey — Thu, 01 Feb 2018 16:08:24 GMT

Hi @joh snow
The version of NiFi used under Kylo is a bit different from the Apache version, but I might be able to give some advice.
Can you tell me which version of Kylo / NiFi you are using, and how many cluster nodes of what kind of hardware you are running on?

Re: Why it costs about 2 hours when we start nifi with 2000 process groups.

hailong_yang — Thu, 01 Feb 2018 17:48:28 GMT

Hi @Dan Chaffelson

Thanks very much. The nifi version which we using is 1.3.0 . The kylo version which we useing is 0.8.4. Our cluster is standalone.

We run it in Linux which have 200G disk and 64G memory.