Gold ⭐️

Solved

Update Attribute processor Not working after upgrade from version 1.8 to version 1.9

9 replies
12421 views
Translate All

Update Attribute processor Not working after upgrade from version 1.8 to version 1.9.

We recently upgraded our Nifi Cluster from version 1.8 to version 1.9 and we were surprised to see the update attribute processor all going invalid for our existing flows. We had to manually fix the existing flow but this ideally should have been handled with the upgrade .

Is there a way to overcome this ?

Apache NiFi

MattWhoAnswer

Titanium ⭐️⭐️⭐️

@Abhinav Joshi

What was the reason given for why the UpdateAttribute processors were now invalid?

MattWhoAnswer

Titanium ⭐️⭐️⭐️

@Abhinav Joshi

What was the reason given for why the UpdateAttribute processors were now invalid?

Translate

MattWho

Titanium ⭐️⭐️⭐️

@Abhinav Joshi

A processor will appear with a dashed line around perimeter of the processor box for a couple reasons

1. It is a ghost implementation --> when loading the flow.xml.gz a processor is encountered that uses a custom nar which does not exits in the current NiFi installation.

2. The user logged into the canvas does not have required permissions to view the component.

-

Based on your description, it does not sound like scenario 2.

The question is why did NiFi 1.9 not find a processor in NiFi 1.9 that used same processor class. Since you upgraded from NiFi 1.8, the update Attribute processor should have referenced the org.apache.nifi.processors.attributes.UpdateAttribute class for version 1.8.x. During an upgrade of NiFi the same identical class would have been found except for a newer version. In this case NiFi would have automatically switched to using the new version.

-

So this raises two questions:

1. Did you customized version of the updateAttribute processor running in NiFi 1.8?

2. Do you have multiple copies of the same processor class but with different versions loaded in your NiFi 1.9?

-

I would suggest inspecting the nifi-app.log from the time of startup immediate following the upgrade. If the UpdateAttribute processors was replaced by a "ghost" processor you would see that logged in the nifi-app.log. This log should help to see why it chose to load a ghost processor.

-

Thank you,

Matt

-

If you found this answer addressed your question, please take a moment to login in and click the "ACCEPT" link.

Translate

MattWho

Titanium ⭐️⭐️⭐️

@Abhinav Joshi

*** Community Forum Tip: Try to avoid starting a new answer in response to an existing answer. Instead use comments to respond to existing answers. There is no guaranteed order to different answer which can make it hard following a discussion.

I would suggest searching the nifi/work directory for multiple versions of the update-attribute nar bundle. You may have multiple nars of different versions installed. The flow.xml.gz file does contain the specific processor version for each component. When starting NiFi 1.9 using the flow.xml.gz from another NiFi version, the component versions will automatically be updated to the new version only if a single option exists. If you have an updateAttribute-1.8.<custom> and an updateAttribute-1.9.0 version available and the flow.xml.gz has an updateAttribute-1.8.0 then it will not auto-update because there are two options and it does not know which should be used.
-

My guess here is that your NiFi 1.8.0 contained both the standard 1.8.0 version of the the updateAttribute processor and a custom version of the updateAttribute processor. Then your flow contained updateAttribute components of each,

Then you upgraded to NiFi 1.9.0 which replaced the stock updateAttribute with 1.9.0 and the custom version of Update Attribute processor was also carried over to your NiFi 1.9.0 install.

-

Thanks,

Matt

Translate

abhinav_joshiAuthor

Gold ⭐️

Hi Matt , for the existing flow , the "update attribute" processor was in an inconfigurable state i.e it had this dotted line all over it and when I clicked on it it wasnt showing anything. This has happened for all the "update attribute" processors in our flow. For now , we have had to delete this processor and add a new one and add configur it.

We also realize that the new "update attribute" processor has an extra feature with version 1.9 which wasnt available in 1.8. However this should be taken care of automatically with an upgrade.

Translate

abhinav_joshiAuthor

Gold ⭐️

HI Matt , Thanks for your answer. I am attaching the screen shot below . You can see how just the " Update Attribute" seems to have those dashed lines and is in "invalid" state. I doubt think our developers made any custom change to the update attribute processor. Let me know what you think about this meanwhile I have looked at the app.log and can see this error

You were bang on when you said about the 'Ghost' Implementation. What do you think should be the next steps

-bash-4.2$ cat nifi-app-*.log |grep "Unable to find bundle for coordinate"

Caused by: java.lang.IllegalStateException: Unable to find bundle for coordinate org.apache.nifi:nifi-update-attribute-nar:1.8.0

ERROR [main] o.a.nifi.controller.ExtensionBuilder Could not create Processor of type org.apache.nifi.processors.attributes.UpdateAttribute for ID 629d355f-0167-1000-ffff-ffffbc497b67; creating "Ghost" implementation

Translate

MattWho

Titanium ⭐️⭐️⭐️

@abhinav_joshi

You should have been able to right click on the "Ghost" processor and select "change version" option. This would have presented you with all the available versions in your NiFi installation. Simply select the one you want to use would resolve your issue.

While this work great when you only have a few ghost processor created from your dataflow, it can be annoying to follow these steps for many components.

The question here is why does you deployment of NiFi have multiple versions of the same NiFi nar installed. NiFi would not ship this way, so that means that additional nar(s) of different versions where added to your NiFi lib directory or to the NiFi extensions directory. You should remove these duplicate nars to avoid running into this issue again.

When only one version exists, dataflow imported/loaded with older versions will auto switch to version used in the NiFi in which dataflow was loaded (this may mean and older or newer version of nar classes).

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Translate

abhinav_joshiAuthor

Gold ⭐️

Thanks Matt . We edited the flow.xml file and removed the reference to 1.4 Update Attribute bundle from it and used the edited flow file to start Nifi and the issue got fixed. No more ghost Update Attribute Processors.

Thanks alot for all your help and detailed explanation on this,

Translate

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.