Streaming analytics manager fail to deploy application because of DependencyResolutionException

[#1]

When i try to deploy a SAM application on HDF 3 enviroment i am getting the following error :

Topology submission failed due to: java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1

SAM log : Exception in thread "main" java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at org.apache.storm.submit.command.DependencyResolverMain.main(DependencyResolverMain.java:86) Caused by: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:387) at org.apache.storm.submit.dependency.DependencyResolver.resolve(DependencyResolver.java:95) at org.apache.storm.submit.command.DependencyResolverMain.main(DependencyResolverMain.java:75) Caused by: org.eclipse.aether.collection.DependencyCollectionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:208) at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:341) ... 2 more Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:329) at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:217) at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:202) ... 3 more Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.apache.kafka:kafka-clients:pom:0.10.2.1 from/to central (http://repo1.maven.org/maven2/😞 repo1.maven.org: Temporary failure in name resolution at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223) at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:314) ... 5 more Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.apache.kafka:kafka-clients:pom:0.10.2.1 from/to central (http://repo1.maven.org/maven2/😞 repo1.maven.org: Temporary failure in name resolution at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:43) at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:355) at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:67) at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:581) at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:249) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:520) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421) ... 8 more Caused by: java.net.UnknownHostException: repo1.maven.org: Temporary failure in name resolution at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) at java.net.InetAddress.getAllByName0(InetAddress.java:1276) at java.net.InetAddress.getAllByName(InetAddress.java:1192) at java.net.InetAddress.getAllByName(InetAddress.java:1126) at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:44) at org.apache.http.impl.conn.DefaultClientConnectionOperator.resolveHostname(DefaultClientConnectionOperator.java:259) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:159) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57) at org.apache.http.impl.client.DecompressingHttpClient.execute(DecompressingHttpClient.java:158) at org.eclipse.aether.transport.http.HttpTransporter.execute(HttpTransporter.java:318) at org.eclipse.aether.transport.http.HttpTransporter.implGet(HttpTransporter.java:274) at org.eclipse.aether.spi.connector.transport.AbstractTransporter.get(AbstractTransporter.java:59) at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask(BasicRepositoryConnector.java:447) at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:350) ... 13 more Traceback (most recent call last): File "/usr/hdf/3.0.0.0-453/storm/bin/storm.py", line 879, in <module> main() File "/usr/hdf/3.0.0.0-453/storm/bin/storm.py", line 876, in main (COMMANDS.get(COMMAND, unknown_command))(*ARGS) File "/usr/hdf/3.0.0.0-453/storm/bin/storm.py", line 290, in jar artifact_to_file_jars = resolve_dependencies(DEP_ARTIFACTS_OPTS, DEP_ARTIFACTS_REPOSITORIES_OPTS) File "/usr/hdf/3.0.0.0-453/storm/bin/storm.py", line 177, in resolve_dependencies raise RuntimeError("dependency handler returns non-zero code: code<%s> syserr<%s>" % (p.returncode, errors)) RuntimeError: dependency handler returns non-zero code: code<1> syserr<None> ERROR [10:16:12.558] [ForkJoinPool-4-worker-13] c.h.s.s.a.t.s.TopologyStates - Error while trying to deploy the topology in the streaming engine java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at com.hortonworks.streamline.streams.actions.storm.topology.StormTopologyActionsImpl.deploy(StormTopologyActionsImpl.java:254) at com.hortonworks.streamline.streams.actions.topology.state.TopologyStates$5.deploy(TopologyStates.java:120) at com.hortonworks.streamline.streams.actions.topology.state.TopologyContext.deploy(TopologyContext.java:87) at com.hortonworks.streamline.streams.actions.topology.service.TopologyActionsService.deployTopology(TopologyActionsService.java:116) at com.hortonworks.streamline.streams.service.TopologyCatalogResource.lambda$deploy$3(TopologyCatalogResource.java:493) at com.hortonworks.streamline.common.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:56) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) ERROR [10:16:12.560] [ForkJoinPool-4-worker-13] c.h.s.c.u.ParallelStreamUtil - Got exception while running async task java.lang.RuntimeException: java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at com.hortonworks.streamline.common.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:58) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) Caused by: java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at com.hortonworks.streamline.streams.actions.storm.topology.StormTopologyActionsImpl.deploy(StormTopologyActionsImpl.java:254) at com.hortonworks.streamline.streams.actions.topology.state.TopologyStates$5.deploy(TopologyStates.java:120) at com.hortonworks.streamline.streams.actions.topology.state.TopologyContext.deploy(TopologyContext.java:87) at com.hortonworks.streamline.streams.actions.topology.service.TopologyActionsService.deployTopology(TopologyActionsService.java:116) at com.hortonworks.streamline.streams.service.TopologyCatalogResource.lambda$deploy$3(TopologyCatalogResource.java:493) at com.hortonworks.streamline.common.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:56) ... 6 common frames omitted

I guess it have something to do with this Sam property :

maven.repo.url : hwx-public^http://repo.hortonworks.com/content/groups/public/,hwx-private^http://nexus-private.hortonworks.com/...

I guess that SAM is trying to connect to maven to download some Dependencies , however i can't have internet access on my server , is there any workaround ?

Thanks in advance

[#2]

Can you predownload or put in a proxy.

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.1/bk_release-notes/content/ch_hdf_relnotes.ht...

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.1/bk_installing-hdf-and-hdp/content/getting_s...

Perhaps build an example code and copy over all the local JARS from .mvn repo

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.1/bk_streaming-analytics-manager-user-guide/c...

[#3]

@Timothy Spann I wish someone could see my post. I think It's a bug.

Based on HDF Release Notes the Storm version is 1.1.0, and this version 1.1.0 does not have proxy URL feature which can be seen in Storm 1.0.4.

From this page (https://github.com/apache/storm/releases), The Release time of Strom v 1.1.0 is earler than Storm 1.0.4.

[#4]

Sorry there was a miss on Storm side to not pulling necessary patch (STORM-2598) to HDF 3.0.0 version line of Storm. The patch is also not released via Apache Storm officially (targeted to 1.2.0).

Please try out workaround below:

1. Please check that there's .m2 directory in 'storm' user's home directory.
   1.a. If it exists, please place artifacts to that directory.
   1.b. If it doesn't exist, go to 2.
2. Please check that there's 'local-repo' in either installation of SAM (streamline) or Storm directory.
   2.a. If it exists, please place artifacts to that directory.
   2.b. Please create .m2 directory in 'storm' user's home directory, and apply 1.a.

I just filed the issue and try to provide a patch shortly.

[#5]

I cannot solve the problem by this method. And please advise. Thanks.

[#6]

Upgrade the the newest version

[#7]

Which version will fix this issue?

[#8]

@yang xiao @Gunar Mueller

I provided the patch and now I believe the fix will be provided to HDF 3.0.2.

You can still try out workaround via my above instruction, but during placing artifacts you need to check artifacts (jar/pom) are placed along with metafile (_remote.repositories), and the repository name is either 'hwx-public' or 'hwx-private' unless it came from maven public repository.

The best bet is pulling artifacts via SAM with internet access, so that artifacts are downloaded to maven local and repository name in _remote.repositories is properly set (important!). Then copies downloaded artifacts (in maven local) to the machines which don’t have internet access.

[#9]

Hi all,
I'm using version 3.1.0 of HDF and I have the exact same issue when following the tutorial, step 3, part 4 "Click on the start button to deploy the topology".

I am using the container version of HDF and my host is a Centos 7 running behind a proxy.

I tried to reproduce the steps described by Jungtaek Lim, but I do not have either any .m2 repository nor a "local-repo" repository in any folder of my container.

I do, however have a "local-repo" file on /tmp/hsperfdata_streamline/local-repo, but I don't understand how I should manually add artefacts to the installation.

Any suggestions would be of great help. Thanks in advance.

Regards,

Yannick

[#10]

We are facing the same issue. Please help.

[#11]

Last two responses are from last year.

Does that still apply to the question asked by @Yannick Bruyninckx two days back?

If not, then can someone from Hortonworks provide an answer as to why this issue is still happening on HDF 3.1.

We are evaluating using HDF 3.1 sandbox and getting exact same issue.

[#12]

I am facing the same issue in latest version of sam as on november 2018, was anyone able to fix this issue?

[#13]

Same problem!

[#14]

I ran into the same problem evaluating the HDF 3.1 sandbox on Ubuntu w/Docker.

According to https://docs.docker.com/network/bridge/#enable-forwarding-from-docker-containers-to-the-outside-worl... the "cda" network that the `docker-deploy-<version>.sh` creates won't have access to the outside world. This would explain why it can't get to repo1.maven.org. I tried running the two suggested commands from the Docker documentation and while I suspect it contributed to the fix, it didn't resolve the problem.

I used the Web Shell Client to login and edit /etc/resolv.conf to change the nameserver to a working one. At that point the topology deployed correctly.

I think as others have suggested, the better solution would be to bake the necessary dependencies into the docker image so that it doesn't need to get to the outside world for the demo to work. A little more sand in the sandbox...

[#15]

how to detemine the artifacts we need ??