Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Streaming analytics manager fail to deploy application because of DependencyResolutionException

Streaming analytics manager fail to deploy application because of DependencyResolutionException

New Contributor

When i try to deploy a SAM application on HDF 3 enviroment i am getting the following error :

Topology submission failed due to: java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1

SAM log : Exception in thread "main" java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at org.apache.storm.submit.command.DependencyResolverMain.main(DependencyResolverMain.java:86) Caused by: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:387) at org.apache.storm.submit.dependency.DependencyResolver.resolve(DependencyResolver.java:95) at org.apache.storm.submit.command.DependencyResolverMain.main(DependencyResolverMain.java:75) Caused by: org.eclipse.aether.collection.DependencyCollectionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:208) at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:341) ... 2 more Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:329) at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:217) at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:202) ... 3 more Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact org.apache.kafka:kafka-clients:pom:0.10.2.1 from/to central (http://repo1.maven.org/maven2/ repo1.maven.org: Temporary failure in name resolution at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223) at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:314) ... 5 more Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact org.apache.kafka:kafka-clients:pom:0.10.2.1 from/to central (http://repo1.maven.org/maven2/ repo1.maven.org: Temporary failure in name resolution at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:43) at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:355) at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:67) at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:581) at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:249) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:520) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421) ... 8 more Caused by: java.net.UnknownHostException: repo1.maven.org: Temporary failure in name resolution at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) at java.net.InetAddress.getAllByName0(InetAddress.java:1276) at java.net.InetAddress.getAllByName(InetAddress.java:1192) at java.net.InetAddress.getAllByName(InetAddress.java:1126) at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:44) at org.apache.http.impl.conn.DefaultClientConnectionOperator.resolveHostname(DefaultClientConnectionOperator.java:259) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:159) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57) at org.apache.http.impl.client.DecompressingHttpClient.execute(DecompressingHttpClient.java:158) at org.eclipse.aether.transport.http.HttpTransporter.execute(HttpTransporter.java:318) at org.eclipse.aether.transport.http.HttpTransporter.implGet(HttpTransporter.java:274) at org.eclipse.aether.spi.connector.transport.AbstractTransporter.get(AbstractTransporter.java:59) at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask(BasicRepositoryConnector.java:447) at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:350) ... 13 more Traceback (most recent call last): File "/usr/hdf/3.0.0.0-453/storm/bin/storm.py", line 879, in <module> main() File "/usr/hdf/3.0.0.0-453/storm/bin/storm.py", line 876, in main (COMMANDS.get(COMMAND, unknown_command))(*ARGS) File "/usr/hdf/3.0.0.0-453/storm/bin/storm.py", line 290, in jar artifact_to_file_jars = resolve_dependencies(DEP_ARTIFACTS_OPTS, DEP_ARTIFACTS_REPOSITORIES_OPTS) File "/usr/hdf/3.0.0.0-453/storm/bin/storm.py", line 177, in resolve_dependencies raise RuntimeError("dependency handler returns non-zero code: code<%s> syserr<%s>" % (p.returncode, errors)) RuntimeError: dependency handler returns non-zero code: code<1> syserr<None> ERROR [10:16:12.558] [ForkJoinPool-4-worker-13] c.h.s.s.a.t.s.TopologyStates - Error while trying to deploy the topology in the streaming engine java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at com.hortonworks.streamline.streams.actions.storm.topology.StormTopologyActionsImpl.deploy(StormTopologyActionsImpl.java:254) at com.hortonworks.streamline.streams.actions.topology.state.TopologyStates$5.deploy(TopologyStates.java:120) at com.hortonworks.streamline.streams.actions.topology.state.TopologyContext.deploy(TopologyContext.java:87) at com.hortonworks.streamline.streams.actions.topology.service.TopologyActionsService.deployTopology(TopologyActionsService.java:116) at com.hortonworks.streamline.streams.service.TopologyCatalogResource.lambda$deploy$3(TopologyCatalogResource.java:493) at com.hortonworks.streamline.common.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:56) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) ERROR [10:16:12.560] [ForkJoinPool-4-worker-13] c.h.s.c.u.ParallelStreamUtil - Got exception while running async task java.lang.RuntimeException: java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at com.hortonworks.streamline.common.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:58) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) Caused by: java.lang.Exception: Topology could not be deployed successfully: storm deploy command failed with Exception in thread "main" java.lang.RuntimeException: org.eclipse.aether.resolution.DependencyResolutionException: Failed to read artifact descriptor for org.apache.kafka:kafka-clients:jar:0.10.2.1 at com.hortonworks.streamline.streams.actions.storm.topology.StormTopologyActionsImpl.deploy(StormTopologyActionsImpl.java:254) at com.hortonworks.streamline.streams.actions.topology.state.TopologyStates$5.deploy(TopologyStates.java:120) at com.hortonworks.streamline.streams.actions.topology.state.TopologyContext.deploy(TopologyContext.java:87) at com.hortonworks.streamline.streams.actions.topology.service.TopologyActionsService.deployTopology(TopologyActionsService.java:116) at com.hortonworks.streamline.streams.service.TopologyCatalogResource.lambda$deploy$3(TopologyCatalogResource.java:493) at com.hortonworks.streamline.common.util.ParallelStreamUtil.lambda$runAsync$0(ParallelStreamUtil.java:56) ... 6 common frames omitted

I guess it have something to do with this Sam property :

maven.repo.url : hwx-public^http://repo.hortonworks.com/content/groups/public/,hwx-private^http://nexus-private.hortonworks.com/...

I guess that SAM is trying to connect to maven to download some Dependencies , however i can't have internet access on my server , is there any workaround ?

Thanks in advance

23498-sam.png

13 REPLIES 13

Re: Streaming analytics manager fail to deploy application because of DependencyResolutionException

Super Guru

Re: Streaming analytics manager fail to deploy application because of DependencyResolutionException

New Contributor

@Timothy Spann I wish someone could see my post. I think It's a bug.

Based on HDF Release Notes the Storm version is 1.1.0, and this version 1.1.0 does not have proxy URL feature which can be seen in Storm 1.0.4.

From this page (https://github.com/apache/storm/releases), The Release time of Strom v 1.1.0 is earler than Storm 1.0.4.

Re: Streaming analytics manager fail to deploy application because of DependencyResolutionException

Cloudera Employee

Sorry there was a miss on Storm side to not pulling necessary patch (STORM-2598) to HDF 3.0.0 version line of Storm. The patch is also not released via Apache Storm officially (targeted to 1.2.0).

Please try out workaround below:

  1. Please check that there's .m2 directory in 'storm' user's home directory.
    1.a. If it exists, please place artifacts to that directory.
    1.b. If it doesn't exist, go to 2.
  2. Please check that there's 'local-repo' in either installation of SAM (streamline) or Storm directory.
    2.a. If it exists, please place artifacts to that directory.
    2.b. Please create .m2 directory in 'storm' user's home directory, and apply 1.a.

I just filed the issue and try to provide a patch shortly.

Re: Streaming analytics manager fail to deploy application because of DependencyResolutionException

New Contributor

I cannot solve the problem by this method. And please advise. Thanks.

Re: Streaming analytics manager fail to deploy application because of DependencyResolutionException

Super Guru

Upgrade the the newest version

Highlighted

Re: Streaming analytics manager fail to deploy application because of DependencyResolutionException

New Contributor

Which version will fix this issue?

Re: Streaming analytics manager fail to deploy application because of DependencyResolutionException

Cloudera Employee

@yang xiao @Gunar Mueller

I provided the patch and now I believe the fix will be provided to HDF 3.0.2.

You can still try out workaround via my above instruction, but during placing artifacts you need to check artifacts (jar/pom) are placed along with metafile (_remote.repositories), and the repository name is either 'hwx-public' or 'hwx-private' unless it came from maven public repository.

The best bet is pulling artifacts via SAM with internet access, so that artifacts are downloaded to maven local and repository name in _remote.repositories is properly set (important!). Then copies downloaded artifacts (in maven local) to the machines which don’t have internet access.

Re: Streaming analytics manager fail to deploy application because of DependencyResolutionException

New Contributor

Hi all,
I'm using version 3.1.0 of HDF and I have the exact same issue when following the tutorial, step 3, part 4 "Click on the start button to deploy the topology".

I am using the container version of HDF and my host is a Centos 7 running behind a proxy.

I tried to reproduce the steps described by Jungtaek Lim, but I do not have either any .m2 repository nor a "local-repo" repository in any folder of my container.

I do, however have a "local-repo" file on /tmp/hsperfdata_streamline/local-repo, but I don't understand how I should manually add artefacts to the installation.

Any suggestions would be of great help. Thanks in advance.

Regards,

Yannick

Re: Streaming analytics manager fail to deploy application because of DependencyResolutionException

New Contributor

We are facing the same issue. Please help.

Don't have an account?
Coming from Hortonworks? Activate your account here