Member since
07-03-2017
5
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 442 | 08-20-2018 08:52 AM |
08-20-2018
08:52 AM
@Frank Gaibler First to your question: In general secure communication over IP is possible. I had a closer look to the issue you described and I have figured out that the nifi-toolkit does not seem to handle the Subject Alternative Names correctly, when you use it for creating the certificates. Right now, the toolkit does not differentiate between DNS names and IP addresses. Normally, you need to declare each subject alternative name as IP address or DNS name like dns:de1000-xxxx,ip:172.18.61.254,.... in your example. But, the nifi-toolkit ignores this syntax and by default everything is declared as DNS name. You can see it in the following screenshot: Here you should get "IP address 192.168.100.120" normally. So from my point of view, it is just a matter of how you create the certificates in the end, because when you use openssl / Java keytool directly for certificate creation, you can create proper certificates following exactly this way. I will follow up internally, but as a workaround you can create the certificates directly using openssl / Java keytool. This should solve the problem.
... View more
08-15-2017
06:37 AM
Now, I added it, but it doesn't solve the issue. Do you know if this could be a problem with the symbolic links? Interesting thing is that even when the submission fails in SAM with this error, afterwards the storm topology is up and running. You can also view it in SAM. The error you find in streamline log, not storm log.
... View more
08-13-2017
10:24 AM
The full stack trace from /var/log/streamline/streamline.log: 11675 [main] WARNo.a.s.StormSubmitter - Error occurred in invoking submitter hook:[org.apache.atlas.storm.hook.StormAtlasHook] java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_131] at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_131] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) ~[?:1.8.0_131] at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_131] at java.lang.Class.forName0(Native Method) ~[?:1.8.0_131] at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_131] at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:361) [storm-core-1.1.0.3.0.0.0-453.jar:1.1.0.3.0.0.0-453] at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:278) [storm-core-1.1.0.3.0.0.0-453.jar:1.1.0.3.0.0.0-453] at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:390) [storm-core-1.1.0.3.0.0.0-453.jar:1.1.0.3.0.0.0-453] at org.apache.storm.flux.Flux.runCli(Flux.java:171) [448dabba7eb011e793c45254006c55a5.jar:?] at org.apache.storm.flux.Flux.main(Flux.java:98) [448dabba7eb011e793c45254006c55a5.jar:?] Exception in thread "main" org.apache.storm.hooks.SubmitterHookException: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:368) at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:278) at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:390) at org.apache.storm.flux.Flux.runCli(Flux.java:171) at org.apache.storm.flux.Flux.main(Flux.java:98) Caused by: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:361) ... 4 more ERROR[18:17:28.844] [ForkJoinPool-4-worker-3] c.h.s.s.a.t.s.TopologyStates -Error while trying to deploy the topology in the streaming engine java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" org.apache.storm.hooks.SubmitterHookException: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook at com.hortonworks.streamline.streams.actions.storm.topology.StormTopologyActionsImpl.deploy(StormTopologyActionsImpl.java:254) at com.hortonworks.streamline.streams.actions.topology.state.TopologyStates$5.deploy(TopologyStates.java:120) at com.hortonworks.streamline.streams.actions.topology.state.TopologyContext.deploy(TopologyContext.java:87) at com.hortonworks.streamline.streams.actions.topology.service.TopologyActionsService.deployTopology(TopologyActionsService.java:116) at com.hortonworks.streamline.streams.service.TopologyCatalogResource.lambda$deploy$3(TopologyCatalogResource.java:493) at com.hortonworks.streamline.common.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:56) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) ERROR[18:17:28.858] [ForkJoinPool-4-worker-3] c.h.s.c.u.ParallelStreamUtil -Got exception while running async task java.lang.RuntimeException: java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" org.apache.storm.hooks.SubmitterHookException: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook at com.hortonworks.streamline.common.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:58) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) Caused by: java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" org.apache.storm.hooks.SubmitterHookException: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook at com.hortonworks.streamline.streams.actions.storm.topology.StormTopologyActionsImpl.deploy(StormTopologyActionsImpl.java:254) at com.hortonworks.streamline.streams.actions.topology.state.TopologyStates$5.deploy(TopologyStates.java:120) at com.hortonworks.streamline.streams.actions.topology.state.TopologyContext.deploy(TopologyContext.java:87) at com.hortonworks.streamline.streams.actions.topology.service.TopologyActionsService.deployTopology(TopologyActionsService.java:116) at com.hortonworks.streamline.streams.service.TopologyCatalogResource.lambda$deploy$3(TopologyCatalogResource.java:493) at com.hortonworks.streamline.common.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:56) ... 6 common frames omitted
... View more
08-11-2017
04:46 PM
Hello Guys, I have setup a standard HDP 2.6.1.0 & HDF 3.0 cluster. I have enabled the Storm hook for Atlas in Storm config using Ambari. When I disable the hook everything works fine. But when the hook is enabled, I can not submit a topology anymore failing with the following error message in SAM UI: Topology submission failed due to: java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" org.apache.storm.hooks.SubmitterHookException: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook I have checked on the Storm and all worker nodes that the following jar files are available in /usr/hdp/current/storm-client/extlib as symlink: lrwxrwxrwx1 root root atlas-plugin-classloader-0.8.0.2.6.1.0-129.jar -> /usr/hdp/current/atlas-server/hook/storm/atlas-plugin-classloader-0.8.0.2.6.1.0-129.jar lrwxrwxrwx1 root root storm-bridge-shim-0.8.0.2.6.1.0-129.jar -> /usr/hdp/current/atlas-server/hook/storm/storm-bridge-shim-0.8.0.2.6.1.0-129.jar This is the result of the OOTB Ambari installation and I have installed it fresh - no upgrade. The java lib path looks like this: java.library.path /usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib:/usr/hdp/current/storm-client/extlib So the required jar files are in the build path, right? Could it be a permission issue? Any ideas? I am deploying the Storm topology using Streaming Analytics Manager running on the same host as Storm. BR, Alexander
... View more
Labels:
08-11-2017
04:42 PM
Hello Guys, I have setup a standard HDP 2.6.1.0 & HDF 3.0 cluster. I have enabled the Storm hook for Atlas in Storm config using Ambari. When I disable the hook everything works fine. But when the hook is enabled, I can not submit a topology anymore failing with the following error message in SAM UI: Topology submission failed due to: java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" org.apache.storm.hooks.SubmitterHookException: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook I have checked on the Storm and all worker nodes that the following jar files are available in /usr/hdp/current/storm-client/extlib as symlink: lrwxrwxrwx1 root root atlas-plugin-classloader-0.8.0.2.6.1.0-129.jar -> /usr/hdp/current/atlas-server/hook/storm/atlas-plugin-classloader-0.8.0.2.6.1.0-129.jar lrwxrwxrwx1 root root storm-bridge-shim-0.8.0.2.6.1.0-129.jar -> /usr/hdp/current/atlas-server/hook/storm/storm-bridge-shim-0.8.0.2.6.1.0-129.jar This is the result of the OOTB Ambari installation and I have installed it fresh - no upgrade. The java lib path looks like this: java.library.path /usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib:/usr/hdp/current/storm-client/extlib So the required jar files are in the build path, right? Could it be a permission issue? Any ideas? I am deploying the Storm topology using Streaming Analytics Manager running on the same host as Storm. BR, Alexander
... View more
Labels:
