Support Questions

Leo3103 · ‎09-09-2024

What am I doing:

I am new to minifi and I want to get a bit workout on how to use minifi and generate a dataflow and send it to Apache NiFi.
After going through the web I understood that first we need to design the flow using the NiFi and download its JSON definition as json.gz format and place it in the conf folder of minifi. Then start the minifi using run-minifi.bat file.
These are things I have done to build the dataflow for MiNiFi:
1. First I have created a pipeline to read from a table present in my Postgres SQL database using a ExecuteSQL processor, Remote Process Group, Input Port and logAttribute Processor.
  
  Input port connected to a LogAttribute to receive the data from the Remote Process Group
2. Second, I have downloaded the ExecuteSQL, Remote Process Group and LogAttribute pipeline as a JSON file and compressed it as flow.json.gz and placed it in the conf folder and started the minifi instance.

The Error I am Facing:

java.lang.Exception: Unable to load flow due to: org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: Cannot set max number of threads to less than 1
	at org.apache.nifi.headless.HeadlessNiFiServer.start(HeadlessNiFiServer.java:194)
	at org.apache.nifi.minifi.StandardMiNiFiServer.start(StandardMiNiFiServer.java:62)
	at org.apache.nifi.minifi.MiNiFi.<init>(MiNiFi.java:141)
	at org.apache.nifi.minifi.MiNiFi.<init>(MiNiFi.java:62)
	at org.apache.nifi.minifi.MiNiFi.main(MiNiFi.java:226)
Caused by: org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.IllegalArgumentException: Cannot set max number of threads to less than 1
	at org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:477)
	at org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:224)
	at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1729)
	at org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:91)
	at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:805)
	at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:526)
	at org.apache.nifi.headless.HeadlessNiFiServer.start(HeadlessNiFiServer.java:170)
	... 4 common frames omitted
Caused by: java.lang.IllegalArgumentException: Cannot set max number of threads to less than 1
	at org.apache.nifi.controller.FlowController.setMaxThreadCount(FlowController.java:1783)
	at org.apache.nifi.controller.FlowController.setMaxTimerDrivenThreadCount(FlowController.java:1766)
	at org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:410)
	... 10 common frames omitted

What did I try to resolve the Error:

I tried adding this property in my minifi.properties but the file is getting reset to the previous properties making this to get erased on every start of minifi instance nifi.flow.engine.threads= 5 .
I tried adding this in my bootstrap.conf file but the error still persists. nifi.minifi.flow.max.concurrent.threads=1.

I'm hoping someone who has encountered this error before or has an idea about how to resolve it can help me please. Thank you in advance.

VidyaSargur · ‎09-09-2024

@Leo3103, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @MattWho @SAMSAL who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Regards,

Vidya Sargur,
Community Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
Community Guidelines
How to use the forum

MattWho · ‎09-09-2024

@Leo3103

It appeasr you are handling the conversion of your NiFi flow Definition for use in your MiNiFi incorrectly as per doscumetation:
https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quic...

You should be downloading your flow definition (json file) via the NiFi UI.
Then you should be renaming that file "flow.json.raw" (no mention of compression here) and place it in the MiNiFi conf directory.

Once you have your flow.json.raw file in the minifi/conf directory, launch that instance of MiNiFi and your dataflow begins automatically.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Leo3103 · ‎09-09-2024

@VidyaSargur @MattWho Thank you for your time and extending your help. Hey Matt I did as you mentioned in the reply and followed the instructions in the quick start too. Now I am facing a new error
cannot invoke "org.apache.nifi.flow.VersionedProcessGroup.getIdentifier()" because "rootGroup" is null

here is the complete error too can you please help me if you have an idea on how to resolve this error.

Command Exception happened during MiNiFi startup
org.apache.nifi.minifi.bootstrap.configuration.ConfigurationChangeException: Unable to perform reload of received configuration change
        at org.apache.nifi.minifi.bootstrap.service.MiNiFiConfigurationChangeListener.handleChange(MiNiFiConfigurationChangeListener.java:103)
        at org.apache.nifi.minifi.bootstrap.command.StartRunner.regenerateFlowConfiguration(StartRunner.java:279)
        at org.apache.nifi.minifi.bootstrap.command.StartRunner.start(StartRunner.java:131)
        at org.apache.nifi.minifi.bootstrap.command.StartRunner.runCommand(StartRunner.java:107)
        at org.apache.nifi.minifi.bootstrap.RunMiNiFi.run(RunMiNiFi.java:133)
        at org.apache.nifi.minifi.bootstrap.RunMiNiFi.main(RunMiNiFi.java:150)
Caused by: java.lang.NullPointerException: Cannot invoke "org.apache.nifi.flow.VersionedProcessGroup.getIdentifier()" because "rootGroup" is null
        at org.apache.nifi.minifi.commons.service.StandardFlowEnrichService.enrichFlow(StandardFlowEnrichService.java:115)
        at org.apache.nifi.minifi.bootstrap.service.MiNiFiConfigurationChangeListener.handleChange(MiNiFiConfigurationChangeListener.java:92)
        ... 5 common frames omitted

Here is my json definition can you please help me out to resolve this error.

{

"flowContents":{

"identifier":"e112dc88-89fb-3b93-ae94-558da67bb792",

"instanceIdentifier":"a3c3c70f-0191-1000-edb6-e2cf3fd3a7d4",

"name":"flow",

"comments":"",

"position":{

"x":-208.0,

"y":152.0

},

"processGroups":[

],

"remoteProcessGroups":[

{

"identifier":"704a77a9-410d-3114-a724-b37bdc242187",

"instanceIdentifier":"a37a0ea6-0191-1000-eea1-3b9e8a9dcc34",

"name":"NiFi Flow",

"comments":"",

"position":{

"x":-280.0,

"y":456.0

},

"targetUri":"http://localhost:8088/nifi",

"targetUris":"http://localhost:8088/nifi",

"communicationsTimeout":"60 sec",

"yieldDuration":"20 sec",

"transportProtocol":"RAW",

"proxyHost":"",

"proxyUser":"",

"inputPorts":[

{

"identifier":"9221b58f-8ff5-340d-ae7b-401cc053abb0",

"instanceIdentifier":"d4493e75-0fbf-34dc-a6ad-f5d633d1c474",

"name":"FromMiNiFi",

"remoteGroupId":"704a77a9-410d-3114-a724-b37bdc242187",

"concurrentlySchedulableTaskCount":1,

"useCompression":false,

"batchSize":{

},

"componentType":"REMOTE_INPUT_PORT",

"targetId":"9d7361b0-0191-1000-2493-b894111e3ef6",

"scheduledState":"ENABLED",

"groupIdentifier":"704a77a9-410d-3114-a724-b37bdc242187"

},

{

"identifier":"cc5997a9-f62f-3764-9a10-d13e424a4008",

"instanceIdentifier":"f2b798e2-88e4-36d8-b5af-5c667f6a2360",

"name":"Copy of receiveSQLData",

"remoteGroupId":"704a77a9-410d-3114-a724-b37bdc242187",

"concurrentlySchedulableTaskCount":1,

"useCompression":false,

"batchSize":{

},

"componentType":"REMOTE_INPUT_PORT",

"targetId":"a37f55f2-0191-1000-19c7-1260366aa0d3",

"scheduledState":"ENABLED",

"groupIdentifier":"704a77a9-410d-3114-a724-b37bdc242187"

}

],

"outputPorts":[

],

"componentType":"REMOTE_PROCESS_GROUP",

"groupIdentifier":"e112dc88-89fb-3b93-ae94-558da67bb792"

}

],

"processors":[

{

"identifier":"d26346ca-8682-3704-ab65-a33435b1b7d2",

"instanceIdentifier":"a38182fb-0191-1000-54f0-10cf9ad54935",

"name":"LogAttribute",

"comments":"",

"position":{

"x":481.0553418104679,

"y":419.9186557338876

},

"type":"org.apache.nifi.processors.standard.LogAttribute",

"bundle":{

"group":"org.apache.nifi",

"artifact":"nifi-standard-nar",

"version":"1.23.2"

},

"properties":{

"Log prefix":null,

"character-set":"UTF-8",

"Log FlowFile Properties":"true",

"Log Level":"info",

"attributes-to-ignore-regex":null,

"Attributes to Ignore":null,

"Attributes to Log":null,

"attributes-to-log-regex":".*",

"Output Format":"Line per Attribute",

"Log Payload":"false"

},

"propertyDescriptors":{

"Log prefix":{

"name":"Log prefix",

"displayName":"Log prefix",

"identifiesControllerService":false,

"sensitive":false

},

"character-set":{

"name":"character-set",

"displayName":"Character Set",

"identifiesControllerService":false,

"sensitive":false

},

"Log FlowFile Properties":{

"name":"Log FlowFile Properties",

"displayName":"Log FlowFile Properties",

"identifiesControllerService":false,

"sensitive":false

},

"Log Level":{

"name":"Log Level",

"displayName":"Log Level",

"identifiesControllerService":false,

"sensitive":false

},

"attributes-to-ignore-regex":{

"name":"attributes-to-ignore-regex",

"displayName":"Attributes to Ignore by Regular Expression",

"identifiesControllerService":false,

"sensitive":false

},

"Attributes to Ignore":{

"name":"Attributes to Ignore",

"displayName":"Attributes to Ignore",

"identifiesControllerService":false,

"sensitive":false

},

"Attributes to Log":{

"name":"Attributes to Log",

"displayName":"Attributes to Log",

"identifiesControllerService":false,

"sensitive":false

},

"attributes-to-log-regex":{

"name":"attributes-to-log-regex",

"displayName":"Attributes to Log by Regular Expression",

"identifiesControllerService":false,

"sensitive":false

},

"Output Format":{

"name":"Output Format",

"displayName":"Output Format",

"identifiesControllerService":false,

"sensitive":false

},

"Log Payload":{

"name":"Log Payload",

"displayName":"Log Payload",

"identifiesControllerService":false,

"sensitive":false

}

},

"style":{

},

"schedulingPeriod":"0 sec",

"schedulingStrategy":"TIMER_DRIVEN",

"executionNode":"ALL",

"penaltyDuration":"30 sec",

"yieldDuration":"1 sec",

"bulletinLevel":"WARN",

"runDurationMillis":25,

"concurrentlySchedulableTaskCount":1,

"autoTerminatedRelationships":[

"success"

],

"scheduledState":"ENABLED",

"retryCount":10,

"retriedRelationships":[

],

"backoffMechanism":"PENALIZE_FLOWFILE",

"maxBackoffPeriod":"10 mins",

"componentType":"PROCESSOR",

"groupIdentifier":"e112dc88-89fb-3b93-ae94-558da67bb792"

},

{

"identifier":"75f885a8-f584-36cf-9d90-b92a7cad5af0",

"instanceIdentifier":"a3644f7e-0191-1000-b197-019d8ad10c41",

"name":"ExecuteSQL",

"comments":"",

"position":{

"x":-104.0,

"y":64.0

},

"type":"org.apache.nifi.processors.standard.ExecuteSQL",

"bundle":{

"group":"org.apache.nifi",

"artifact":"nifi-standard-nar",

"version":"1.23.2"

},

"properties":{

"esql-max-rows":"0",

"dbf-default-precision":"10",

"Max Wait Time":"0 seconds",

"Database Connection Pooling Service":"822e5ffc-8f36-3466-8ecc-5f319e4980c8",

"sql-post-query":null,

"esql-auto-commit":"true",

"dbf-user-logical-types":"false",

"dbf-default-scale":"0",

"sql-pre-query":null,

"compression-format":"NONE",

"esql-output-batch-size":"0",

"esql-fetch-size":"0",

"SQL select query":"select col1, col2, col3, col4, col5 from public.tjavatest;",

"dbf-normalize":"false"

},

"propertyDescriptors":{

"esql-max-rows":{

"name":"esql-max-rows",

"displayName":"Max Rows Per Flow File",

"identifiesControllerService":false,

"sensitive":false

},

"dbf-default-precision":{

"name":"dbf-default-precision",

"displayName":"Default Decimal Precision",

"identifiesControllerService":false,

"sensitive":false

},

"Max Wait Time":{

"name":"Max Wait Time",

"displayName":"Max Wait Time",

"identifiesControllerService":false,

"sensitive":false

},

"Database Connection Pooling Service":{

"name":"Database Connection Pooling Service",

"displayName":"Database Connection Pooling Service",

"identifiesControllerService":true,

"sensitive":false

},

"sql-post-query":{

"name":"sql-post-query",

"displayName":"SQL Post-Query",

"identifiesControllerService":false,

"sensitive":false

},

"esql-auto-commit":{

"name":"esql-auto-commit",

"displayName":"Set Auto Commit",

"identifiesControllerService":false,

"sensitive":false

},

"dbf-user-logical-types":{

"name":"dbf-user-logical-types",

"displayName":"Use Avro Logical Types",

"identifiesControllerService":false,

"sensitive":false

},

"dbf-default-scale":{

"name":"dbf-default-scale",

"displayName":"Default Decimal Scale",

"identifiesControllerService":false,

"sensitive":false

},

"sql-pre-query":{

"name":"sql-pre-query",

"displayName":"SQL Pre-Query",

"identifiesControllerService":false,

"sensitive":false

},

"compression-format":{

"name":"compression-format",

"displayName":"Compression Format",

"identifiesControllerService":false,

"sensitive":false

},

"esql-output-batch-size":{

"name":"esql-output-batch-size",

"displayName":"Output Batch Size",

"identifiesControllerService":false,

"sensitive":false

},

"esql-fetch-size":{

"name":"esql-fetch-size",

"displayName":"Fetch Size",

"identifiesControllerService":false,

"sensitive":false

},

"SQL select query":{

"name":"SQL select query",

"displayName":"SQL select query",

"identifiesControllerService":false,

"sensitive":false

},

"dbf-normalize":{

"name":"dbf-normalize",

"displayName":"Normalize Table/Column Names",

"identifiesControllerService":false,

"sensitive":false

}

},

"style":{

},

"schedulingPeriod":"0 sec",

"schedulingStrategy":"TIMER_DRIVEN",

"executionNode":"ALL",

"penaltyDuration":"30 sec",

"yieldDuration":"1 sec",

"bulletinLevel":"WARN",

"runDurationMillis":0,

"concurrentlySchedulableTaskCount":1,

"autoTerminatedRelationships":[

],

"scheduledState":"ENABLED",

"retryCount":10,

"retriedRelationships":[

],

"backoffMechanism":"PENALIZE_FLOWFILE",

"maxBackoffPeriod":"10 mins",

"componentType":"PROCESSOR",

"groupIdentifier":"e112dc88-89fb-3b93-ae94-558da67bb792"

}

],

"inputPorts":[

],

"outputPorts":[

],

"connections":[

{

"identifier":"a4c0a496-5351-3cc3-a4fb-4aadcfb93ac4",

"instanceIdentifier":"a3819773-0191-1000-47c5-0adb24a80f96",

"name":"",

"source":{

"id":"75f885a8-f584-36cf-9d90-b92a7cad5af0",

"type":"PROCESSOR",

"groupId":"e112dc88-89fb-3b93-ae94-558da67bb792",

"name":"ExecuteSQL",

"comments":"",

"instanceIdentifier":"a3644f7e-0191-1000-b197-019d8ad10c41"

},

"destination":{

"id":"d26346ca-8682-3704-ab65-a33435b1b7d2",

"type":"PROCESSOR",

"groupId":"e112dc88-89fb-3b93-ae94-558da67bb792",

"name":"LogAttribute",

"comments":"",

"instanceIdentifier":"a38182fb-0191-1000-54f0-10cf9ad54935"

},

"labelIndex":1,

"zIndex":0,

"selectedRelationships":[

"failure"

],

"backPressureObjectThreshold":10000,

"backPressureDataSizeThreshold":"1 GB",

"flowFileExpiration":"0 sec",

"prioritizers":[

],

"bends":[

],

"loadBalanceStrategy":"DO_NOT_LOAD_BALANCE",

"partitioningAttribute":"",

"loadBalanceCompression":"DO_NOT_COMPRESS",

"componentType":"CONNECTION",

"groupIdentifier":"e112dc88-89fb-3b93-ae94-558da67bb792"

},

{

"identifier":"51ef2902-0ce4-3254-8398-0f07b666ec6b",

"instanceIdentifier":"a380e710-0191-1000-5968-000a66ac2604",

"name":"",

"source":{

"id":"75f885a8-f584-36cf-9d90-b92a7cad5af0",

"type":"PROCESSOR",

"groupId":"e112dc88-89fb-3b93-ae94-558da67bb792",

"name":"ExecuteSQL",

"comments":"",

"instanceIdentifier":"a3644f7e-0191-1000-b197-019d8ad10c41"

},

"destination":{

"id":"cc5997a9-f62f-3764-9a10-d13e424a4008",

"type":"REMOTE_INPUT_PORT",

"groupId":"704a77a9-410d-3114-a724-b37bdc242187",

"name":"Copy of receiveSQLData",

"instanceIdentifier":"f2b798e2-88e4-36d8-b5af-5c667f6a2360"

},

"labelIndex":1,

"zIndex":0,

"selectedRelationships":[

"success"

],

"backPressureObjectThreshold":10000,

"backPressureDataSizeThreshold":"1 GB",

"flowFileExpiration":"0 sec",

"prioritizers":[

],

"bends":[

],

"loadBalanceStrategy":"DO_NOT_LOAD_BALANCE",

"partitioningAttribute":"",

"loadBalanceCompression":"DO_NOT_COMPRESS",

"componentType":"CONNECTION",

"groupIdentifier":"e112dc88-89fb-3b93-ae94-558da67bb792"

}

],

"labels":[

],

"funnels":[

],

"controllerServices":[

],

"variables":{

},

"defaultFlowFileExpiration":"0 sec",

"defaultBackPressureObjectThreshold":10000,

"defaultBackPressureDataSizeThreshold":"1 GB",

"componentType":"PROCESS_GROUP",

"flowFileOutboundPolicy":"STREAM_WHEN_AVAILABLE",

"flowFileConcurrency":"UNBOUNDED"

},

"externalControllerServices":{

"822e5ffc-8f36-3466-8ecc-5f319e4980c8":{

"identifier":"822e5ffc-8f36-3466-8ecc-5f319e4980c8",

"name":"DBCPConnectionPool"

}

},

"parameterContexts":[],

"flowEncodingVersion":"1.0",

"parameterProviders":[],

"latest":false

}

MattWho · ‎09-11-2024

@Leo3103

This exception might be caused by a missing nar file in your MiNiFi.
MiNiFi does not include all the NiFi nars.
https://github.com/apache/nifi/blob/main/minifi/minifi-docs/src/main/markdown/minifi-java-agent-quic...

A quick look through your flow definition I saw you were using the ExecuteSQL processor. While ExecuteSQL is part of one of the included nars, it does require another nar that is not included (this is noted in the above linked quick start guide).

You'll need to add the "nifi-dbcp-service-nar" from your NiFi distribution to your MiNiFi's lib directory.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Leo3103 · ‎09-12-2024

@MattWho thank you for responding back, I do understand I did not include the "nifi-dbcp-service-nar". But this time I am trying to run a new flow which is defined as below

Image 1

Image 2: This the flow designed inside the internalTest processGroup

When I downloaded the flow definition of the internalTest and placed it in the conf folder of minifi I am still facing the same issue.

Leo3103 · ‎09-25-2024

@MattWho I did try adding the "nifi-dbcp-service-nar" from NiFi distribution to MiNiFi's lib directory. Even then I am facing the same error

C:\SoftwareCenterApplications\minifi-2.0.0-M4-bin\minifi-2.0.0-M4\bin>run-minifi.bat
2024-09-25 18:00:58,785 INFO [main] o.a.n.minifi.properties.PropertiesLoader Loading Bootstrap Properties [conf\bootstrap.conf]
2024-09-25 18:00:58,911 INFO [main] o.a.n.minifi.properties.PropertiesLoader Loading Bootstrap Properties [conf\bootstrap.conf]
2024-09-25 18:00:58,917 INFO [main] o.a.n.minifi.properties.PropertiesLoader Loading Bootstrap Properties [conf\bootstrap.conf]
2024-09-25 18:00:59,047 INFO [main] o.a.n.minifi.properties.PropertiesLoader Loading Bootstrap Properties [conf\bootstrap.conf]
2024-09-25 18:00:59,612 ERROR [main] o.apache.nifi.minifi.bootstrap.Command Exception happened during MiNiFi startup
org.apache.nifi.minifi.bootstrap.configuration.ConfigurationChangeException: Unable to perform reload of received configuration change
        at org.apache.nifi.minifi.bootstrap.service.MiNiFiConfigurationChangeListener.handleChange(MiNiFiConfigurationChangeListener.java:103)
        at org.apache.nifi.minifi.bootstrap.command.StartRunner.regenerateFlowConfiguration(StartRunner.java:279)
        at org.apache.nifi.minifi.bootstrap.command.StartRunner.start(StartRunner.java:131)
        at org.apache.nifi.minifi.bootstrap.command.StartRunner.runCommand(StartRunner.java:107)
        at org.apache.nifi.minifi.bootstrap.RunMiNiFi.run(RunMiNiFi.java:133)
        at org.apache.nifi.minifi.bootstrap.RunMiNiFi.main(RunMiNiFi.java:150)
Caused by: java.lang.NullPointerException: Cannot invoke "org.apache.nifi.flow.VersionedProcessGroup.getIdentifier()" because "rootGroup" is null
        at org.apache.nifi.minifi.commons.service.StandardFlowEnrichService.enrichFlow(StandardFlowEnrichService.java:115)
        at org.apache.nifi.minifi.bootstrap.service.MiNiFiConfigurationChangeListener.handleChange(MiNiFiConfigurationChangeListener.java:92)
        ... 5 common frames omitted

Can you please help me out if you have any idea on this. Thank you in advance.

MattWho · ‎09-30-2024

@Leo3103

Things to note.

Your process group that you will be exporting a flow definition for can NOT contain local input or output ports at the top level. ---> The Process Group will become the "root" process group in your MiNiFi deployment. Only "remote" input or output ports are supported at the root level. Within the Process group you are exporting, you can have child process group deeper down that contain local input and output ports. But the test process group "internalTest" you are exporting has a local output port.
If you take entire flow.json.gz from the NiFi conf directory and rename it to flow.json.raw, you can start MiNiFi with no issues.
Since you are exporting a flow definition of a process group, you'll need to utilize the MiNiFi toolkit to transform it in to the proper format that can be loaded by MiNiFi. The MiNiFi-toolkit can be downloaded from here: https://nifi.apache.org/download/ (select "MINIFI' and click download link for Toolkit). Execute:

./minifi-toolkit/bin/config.sh transform-nifi <exported flow definition> flow.json.raw

Now edit the flow.json.raw file and edit the following property at start of file (value can not be 0.)

"maxTimerDrivenThreadCount":5

Now you can start your MiNiFi and it will create the flow.json.gz as it starts.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

MattWho · ‎09-30-2024

@Leo3103

Here are some additional challenges you may have:

NiFi flow definitions do not export with any encrypted values. So it your dataflow uses a component that has sensitive properties (passwords), those will not exist in the flow.json you export. You could get the encrypted values ( they will look like enc{....<long string>....}) from the NiFi flow.json.gz file and add them in to your flow.json.raw produced by the toolkit.
In order for your MiNiFi to be able to load a flow.json.raw containing sensitive encrypted values, the same sensitive.props.key value and sensitive.props.algorithm used in your NiFi (nifi.properties) must be used in your MiNiFi bootstrap.conf.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Leo3103 · ‎10-08-2024

Hi @MattWho, once again Thank you as always.

After going through your answer these are things I did.

1) I have created a new flow as attached below

Image 1

Image 2

Where Image 1 represents the complete flow and Image 2 shows the flow inside the remoteMiNiFiTest.

2) After designing the flow I have download the flow of the remoteMiNiFiTest as a .json file and converted into flow.json.gz using the minifi toolkit. ( Why I didn't convert it into a flow.json.raw

(2.a) Because I am facing the same error as mentioned in the above thread

"org.apache.nifi.flow.VersionedProcessGroup.getIdentifier()" because "rootGroup" is null

(2.b) While running the run-minifi.bat file the instance of minifi is not starting up and prompting that the format of the flow configuration file is not in the format of GZIP ).

3) So in - order to tackle these errors I have converted it into flow.json.gz rather flow.json.raw.

4) Even though after my minifi is getting started up without any issues or breakdowns due to exception thrown by errors, the flowfile data is not getting generated by minifi.

Can you please help me out if you know what all wrong I am doing if you have an idea or so. Thank you for responding for all the previous troubles.

Support Questions

Unable to start MiNiFi because of an error caused due to the number of threads

What am I doing:

The Error I am Facing:

What did I try to resolve the Error: