Support Questions
Find answers, ask questions, and share your expertise

Elasticsearch classes conflict in Nifi

Expert Contributor

Hello Experts,

 

I have implemented a custom log appender which inserts log messages to ElasticSearch.

Custom appender class works fine when it is run independently as executable jar.

 

But when I add this custom appender jar file to nifi "lib" directory, nifi stopped working - its not able to start.

I have below 2 dependencies in my appender jar -

<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.15.0</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.15.0</version>
</dependency>

 

And Nifi has multiple processors related to ElasticSearch so the nar files of these processors also have same classes which are present in above 2 dependencies.
This looks to be the issue.

 

I need these ElasticSearch dependencies classes at bootstrap level for logger to get initialised.
But the nar files get loaded at later point, so I cant depend on the classes which are part of nar files.

 

What is the better approach to solve this issue ?

Any suggestion would be much appreciated.

 

 

Thanks

Mahendra

2 REPLIES 2

Expert Contributor

I tried to change the package name of these 2 dependencies with the help of maven shade plugin, shade has places these dependencies classes in new package.

Still whenever I add any of these dependenies to maven/jar nifi is failing to start !!!

Expert Contributor

Below is the app log - 

 

2021-11-18 16:07:13,819 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider org.apache.nifi.processors.windows.event.log.ConsumeWindowsEventLog could not be instantiated
java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider org.apache.nifi.processors.windows.event.log.ConsumeWindowsEventLog could not be instantiated
at java.util.ServiceLoader.fail(ServiceLoader.java:232)
at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at org.apache.nifi.nar.StandardExtensionDiscoveringManager.loadExtensions(StandardExtensionDiscoveringManager.java:156)
at org.apache.nifi.nar.StandardExtensionDiscoveringManager.discoverExtensions(StandardExtensionDiscoveringManager.java:131)
at org.apache.nifi.nar.StandardExtensionDiscoveringManager.discoverExtensions(StandardExtensionDiscoveringManager.java:117)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1042)
at org.apache.nifi.NiFi.<init>(NiFi.java:158)
at org.apache.nifi.NiFi.<init>(NiFi.java:72)
at org.apache.nifi.NiFi.main(NiFi.java:301)
Caused by: java.lang.NoSuchFieldError: SIZE
at com.sun.jna.platform.win32.WinBase.<clinit>(WinBase.java:52)
at com.sun.jna.platform.win32.Kernel32Util.getComputerName(Kernel32Util.java:60)
at org.apache.nifi.processors.windows.event.log.ConsumeWindowsEventLog.<init>(ConsumeWindowsEventLog.java:178)
at org.apache.nifi.processors.windows.event.log.ConsumeWindowsEventLog.<init>(ConsumeWindowsEventLog.java:164)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
... 9 common frames omitted
2021-11-18 16:07:13,820 INFO [Thread-19] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2021-11-18 16:07:13,820 INFO [Thread-19] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).