Support Questions

Find answers, ask questions, and share your expertise

Looking for a Tool to Simplify Apache NiFi Flow Deployments Across Multiple Environments – Any Suggestions?

avatar
New Contributor

Hi everyone, I need some quick help. I was doing some research on a tool that can help streamline the deployment of Apache NiFi flows across multiple environments. Does anyone know of any such tools, whether free or paid?

1 ACCEPTED SOLUTION

avatar
Master Mentor

@David07 

Welcome to the community.

NiFi-Registry would allow you to version control NiFi Process Group dataflows.  You can then connect multiple NiFi instances/cluster to the same NiFi Registry which would allow those other NiFi's access to these version controlled flow definitions (if authorized correctly).   

From within NiFi, you can also download process groups as flow definition json files.  You can use these to create offline catalog of these flow definitions for ease of reuse in other NiFi instances/clusters.  You can easily import a flow definitions to the canvas of a NiFi instance.  For more details here, read the "Building a Dataflow" section of the NiFi User Guide.  Importing and downloading flow definitions is covered in the "Process Group" section.

Tip: Building your dataflows using parameter contexts for properties that may have unique values per environments (URL, passwords, usernames, etc.) makes the process of sharing or moving flow definitions between NiFi deployments.  Each environment may have different values assigned to the parameter contexts referenced in NiFi processors.

Cloudera offers a unique option for rapid multiple NiFi instance deployments through Cloudera Edge Flow Manager.   This option requires a license with Cloudera to download this product.  This tool allows you to construct a dataflows just how you would in NiFi and then deploy that dataflow to one or more agents (MiNiFi instances).  This provides you with a central management location for managing multiple unique dataflow deployments to unique agents.

Please help our community grow and trhive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

View solution in original post

2 REPLIES 2

avatar
Master Mentor

@David07 

Welcome to the community.

NiFi-Registry would allow you to version control NiFi Process Group dataflows.  You can then connect multiple NiFi instances/cluster to the same NiFi Registry which would allow those other NiFi's access to these version controlled flow definitions (if authorized correctly).   

From within NiFi, you can also download process groups as flow definition json files.  You can use these to create offline catalog of these flow definitions for ease of reuse in other NiFi instances/clusters.  You can easily import a flow definitions to the canvas of a NiFi instance.  For more details here, read the "Building a Dataflow" section of the NiFi User Guide.  Importing and downloading flow definitions is covered in the "Process Group" section.

Tip: Building your dataflows using parameter contexts for properties that may have unique values per environments (URL, passwords, usernames, etc.) makes the process of sharing or moving flow definitions between NiFi deployments.  Each environment may have different values assigned to the parameter contexts referenced in NiFi processors.

Cloudera offers a unique option for rapid multiple NiFi instance deployments through Cloudera Edge Flow Manager.   This option requires a license with Cloudera to download this product.  This tool allows you to construct a dataflows just how you would in NiFi and then deploy that dataflow to one or more agents (MiNiFi instances).  This provides you with a central management location for managing multiple unique dataflow deployments to unique agents.

Please help our community grow and trhive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

avatar
Community Manager

@David07 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  Thanks.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: