Hi,
We're using the Hortonworks repository to build our Hadoop project using Maven. For our project, one of the dependencies is on hive-exec JAR file. The version in HDP 3.1.4 for hive-exec in our HDP distribution is 3.1.0.3.1.4.0-315 and the dependency is specified as:
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>3.1.0.3.1.4.0-315</version>
</dependency>
As Maven resolves all transitive dependencies automatically, this JAR file is giving the following error:
Could not find artifact org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.3.2.6.5.0-SNAPSHOT in hortonworks (http://repo.hortonworks.com/content/groups/public)
By viewing the dependency tree for hive-exec, this specific snippet is related to the error:
org.apache.hive:hive-exec:jar:3.1.0.3.1.4.0-315:provided
org.apache.hive:hive-shims:jar:3.1.0.3.1.4.0-315:provided
org.apache.hive.shims:hive-shims-common:jar:3.1.0.3.1.4.0-315:provided
org.apache.hive.shims:hive-shims-0.20:jar:3.1.0.3.1.4.0-315:provided
org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.3.2.6.5.0-SNAPSHOT:provided
The version 2.7.3.2.6.5.0-SNAPSHOT for hadoop-yarn-server-resourcemanager is specified in the parent POM of the Hive project within the Hortonworks repository. From the error, and from own research, this SNAPSHOT version does not exist in any repository, Hortonworks, Maven Central or Apache-Snapshot repository which leads to the build error. The version number is obtained from the property defined in the Apache Hive parent pom:
<hadoop2.version>2.7.3.2.6.5.0-SNAPSHOT</hadoop2.version>
Prior version of HDP (3.1.0), hive-exec had a dependency on hive-shims-0.23 with the dependency tree of:
org.apache.hive:hive-exec:jar:3.1.0.3.1.0.0-78:provided
org.apache.hive:hive-shims:jar:3.1.0.3.1.0.0-78:provided
org.apache.hive.shims:hive-shims-common:jar:3.1.0.3.1.0.0-78:provided
org.apache.hive.shims:hive-shims-0.23:jar:33.1.0.3.1.0.0-78:provided
org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.1.1.3.1.0.0-78:provided
It looks like hive-shims version was lowered in HDP 3.1.4 and the hadoop-yarn-server-resourcemanager versions specified does not exist. Since the transitive dependencies are not resolved as per what's provided in the POM files for the projects/dependencies in the Hortonworks repostiory, is this an issue with HDP 3.1.4? Or is there something I can do in my own POM file to resolve this?
Thanks.