Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive Maven Error in HDP 3.1.4

Hive Maven Error in HDP 3.1.4

New Contributor

Hi,

We're using the Hortonworks repository to build our Hadoop project using Maven. For our project, one of the dependencies is on hive-exec JAR file. The version in HDP 3.1.4 for hive-exec in our HDP distribution is 3.1.0.3.1.4.0-315 and the dependency is specified as:

 

 

        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-exec</artifactId>
            <version>3.1.0.3.1.4.0-315</version>
        </dependency>

 

 

As Maven resolves all transitive dependencies automatically, this JAR file is giving the following error:

Could not find artifact org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.3.2.6.5.0-SNAPSHOT in hortonworks (http://repo.hortonworks.com/content/groups/public)

 

By viewing the dependency tree for hive-exec, this specific snippet is related to the error:

 

 

org.apache.hive:hive-exec:jar:3.1.0.3.1.4.0-315:provided
   org.apache.hive:hive-shims:jar:3.1.0.3.1.4.0-315:provided
          org.apache.hive.shims:hive-shims-common:jar:3.1.0.3.1.4.0-315:provided
                org.apache.hive.shims:hive-shims-0.20:jar:3.1.0.3.1.4.0-315:provided
                      org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:2.7.3.2.6.5.0-SNAPSHOT:provided

 

 

The version 2.7.3.2.6.5.0-SNAPSHOT for hadoop-yarn-server-resourcemanager is specified in the parent POM of the Hive project within the Hortonworks repository. From the error, and from own research, this SNAPSHOT version does not exist in any repository, Hortonworks, Maven Central or Apache-Snapshot repository which leads to the build error. The version number is obtained from the property defined in the Apache Hive parent pom:

 

 

<hadoop2.version>2.7.3.2.6.5.0-SNAPSHOT</hadoop2.version>

 

 

Prior version of HDP (3.1.0), hive-exec had a dependency on hive-shims-0.23 with the dependency tree of:

 

 

org.apache.hive:hive-exec:jar:3.1.0.3.1.0.0-78:provided
   org.apache.hive:hive-shims:jar:3.1.0.3.1.0.0-78:provided
          org.apache.hive.shims:hive-shims-common:jar:3.1.0.3.1.0.0-78:provided
                org.apache.hive.shims:hive-shims-0.23:jar:33.1.0.3.1.0.0-78:provided
                      org.apache.hadoop:hadoop-yarn-server-resourcemanager:jar:3.1.1.3.1.0.0-78:provided

 

 

It looks like hive-shims version was lowered in HDP 3.1.4 and the hadoop-yarn-server-resourcemanager versions specified does not exist. Since the transitive dependencies are not resolved as per what's provided in the POM files for the projects/dependencies in the Hortonworks repostiory, is this an issue with HDP 3.1.4? Or is there something I can do in my own POM file to resolve this?

 

Thanks.

1 REPLY 1
Highlighted

Re: Hive Maven Error in HDP 3.1.4

New Contributor

I was able to get around this by excluding the transitive dependencies since I only needed the hive-exec file to compile the project. However, I still think that the project file should be fixed in the Hortonworks repo as it's pointing to an artifact that does not exist in any public repositories.

Don't have an account?
Coming from Hortonworks? Activate your account here