question Re: nifi 2.0 bug with InvokeHTTP procdessor in Support Questions

nifi 2.0 bug with InvokeHTTP procdessor

arutkwccu — Tue, 02 Jan 2024 18:58:39 GMT

There is a bug with the InvokeHTTP processor in version 2.0. I carried over my flows from version 1.23, which I upgraded to 1.24, and then I exported the flows as processing groups and imported them into version 2.0. These flows worked fine, previously. In one, I have an InvokeHTTP processor that is sending the document to a Tika server for processing. In version 2.0 there is a new property in this processor that was added. I did not put it there. The documentation makes no mention of it.

The property was `Proxy Type` and it's value was `http`. I tried to set it as an empty string, but I continued to get a failure. The error I was getting was:
18:17:26 UTC ERROR InvokeHTTP[id=dfae880c-b84a-3f2c-f1d9-941b8fc625eb] Request Processing failed: FlowFile[filename=random_20210909a.docx]: java.lang.IllegalArgumentException: Unexpected char 0x20 at 5 in header name: Proxy Type
Even after setting this property to an empty string, the error persisted.

I remedied the situation by deleting the property altogether. After deleting the property, the flow continued to work as before.

Re: nifi 2.0 bug with InvokeHTTP procdessor

joseomjr — Tue, 02 Jan 2024 19:26:11 GMT

If you were able to delete it, then it almost sounds like that attribute might have been something from the 1.X versions and not something new in the 2.X version.

Re: nifi 2.0 bug with InvokeHTTP procdessor

MattWho — Tue, 02 Jan 2024 20:43:38 GMT

@arutkwccu @joseomjr

NiFi components evolve from release to release. as part of their evolution the components may have default properties added or removed.

That is the case here with invokeHTTP processor. The property "Proxy Type" idd exist as a property in Apache NiFi 1.24 invokeHTTP processor and had a default value of "http".

This property was removed in Apache NIFi 2.0.0-M1 invokeHTTP processor.

In this specific example:
- Earlier version of Apache NIFi 1.x invokeHTTP processor supported configuring proxy properties directly in the processor. Later version of Apache NiFi allowed those same configuration to be abstracted to a StandardProxyConfigurationService controller service (for ease of reuse), but also kept the original direct properties in the processor to avoid upgrade issues for user upgrading within a minor release change. As Apache NiFi 2.0.0-M1 is a major release, clean-up is happening here to remove the deprecated config methods for the newer controller service config method.

When a NiFi loads the flow.json.gz from another NiFi it loads the same class and reads in all the configured properties. For any configured property where the current version does not have the same property, the property is added as a custom dynamic property. This may render your component invalid until manual correction is made (in your case deleting this dynamic property). Apache NiFi has always functioned in this way and it is not a bug. It is not possible for NiFi framework to know what "dynamic" properties should or should not be added to component, it simply reads what is found in the flow.json.gz being loaded. Validating your dataflows post any upgrade (not just major version changes like Apache NiFi 1.x to 2.x) is a necessary step.

This specific migration is mentioned in the Migrating Deprecated Components and Features guide.

You'll know right away if a property is not a default property but rather a "dynamic" custom property when you see it. Only dynamic properties will present the trashcan icon to the right of it allowing you to delete it.

Hope this adds clarity here.

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Mat

Re: nifi 2.0 bug with InvokeHTTP procdessor

arutkwccu — Tue, 02 Jan 2024 20:58:45 GMT

Thank you for the advisement. I understand what you are saying.