Failed to compile HDP hive 2.5.0.0 and 2.4.2.40 with the same error.

onlyforcloud — Sat, 10 Dec 2016 11:40:49 GMT

I failed to compile HDP hive because there isn't any jar including org.apache.hadoop.ipc.CallerContext.

I tried to search this class from apache hadoop 2.7.3 and apache hive 2.1.0 source, but nothing was found.

So, I want to know the details you compiled it.

The compile information:

[INFO] ------------------------------------------------------------------------
[INFO] Building Hive Shims 0.23 1.2.1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-shims-0.23 ---
[INFO] Deleting /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/target
[INFO] Deleting /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23 (includes = [datanucleus.log, derby.log], excludes = [])
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @ hive-shims-0.23 ---
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-shims-0.23 ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-shims-0.23 ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-shims-0.23 ---
[INFO] Executing tasks


main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-shims-0.23 ---
[INFO] Compiling 6 source files to /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/target/classes
[INFO] -------------------------------------------------------------
[WARNING] COMPILATION WARNING : 
[INFO] -------------------------------------------------------------
[WARNING] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java: /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java uses or overrides a deprecated API.
[WARNING] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java: Recompile with -Xlint:deprecation for details.
[WARNING] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java: /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java uses unchecked or unsafe operations.
[WARNING] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java: Recompile with -Xlint:unchecked for details.
[INFO] 4 warnings 
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[70,29] cannot find symbol
  symbol:   class CallerContext
  location: package org.apache.hadoop.ipc
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1405,47] package CallerContext does not exist
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1405,5] cannot find symbol
  symbol:   variable CallerContext
  location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1420,9] cannot find symbol
  symbol:   variable CallerContext
  location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1423,12] cannot find symbol
  symbol:   variable CallerContext
  location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[INFO] 5 errors 
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Hive ............................................... SUCCESS [  4.679 s]
[INFO] Hive Shims Common .................................. SUCCESS [  6.317 s]
[INFO] Hive Shims 0.20S ................................... SUCCESS [  1.946 s]
[INFO] Hive Shims 0.23 .................................... FAILURE [  8.511 s]
[INFO] Hive Shims Scheduler ............................... SKIPPED
[INFO] Hive Shims ......................................... SKIPPED
[INFO] Hive Common ........................................ SKIPPED
[INFO] Hive Serde ......................................... SKIPPED
[INFO] Hive Metastore ..................................... SKIPPED
[INFO] Hive Ant Utilities ................................. SKIPPED
[INFO] Spark Remote Client ................................ SKIPPED
[INFO] Hive Query Language ................................ SKIPPED
[INFO] Hive Service ....................................... SKIPPED
[INFO] Hive Accumulo Handler .............................. SKIPPED
[INFO] Hive JDBC .......................................... SKIPPED
[INFO] Hive Beeline ....................................... SKIPPED
[INFO] Hive CLI ........................................... SKIPPED
[INFO] Hive Contrib ....................................... SKIPPED
[INFO] Hive HBase Handler ................................. SKIPPED
[INFO] Hive HCatalog ...................................... SKIPPED
[INFO] Hive HCatalog Core ................................. SKIPPED
[INFO] Hive HCatalog Pig Adapter .......................... SKIPPED
[INFO] Hive HCatalog Server Extensions .................... SKIPPED
[INFO] Hive HCatalog Webhcat Java Client .................. SKIPPED
[INFO] Hive HCatalog Webhcat .............................. SKIPPED
[INFO] Hive HCatalog Streaming ............................ SKIPPED
[INFO] Hive HWI ........................................... SKIPPED
[INFO] Hive ODBC .......................................... SKIPPED
[INFO] Hive Shims Aggregator .............................. SKIPPED
[INFO] Hive TestUtils ..................................... SKIPPED
[INFO] Hive Packaging ..................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 22.817 s
[INFO] Finished at: 2016-12-10T12:51:57+08:00
[INFO] Final Memory: 49M/306M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hive-shims-0.23: Compilation failure: Compilation failure:
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[70,29] cannot find symbol
[ERROR] symbol:   class CallerContext
[ERROR] location: package org.apache.hadoop.ipc
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1405,47] package CallerContext does not exist
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1405,5] cannot find symbol
[ERROR] symbol:   variable CallerContext
[ERROR] location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1420,9] cannot find symbol
[ERROR] symbol:   variable CallerContext
[ERROR] location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[ERROR] /root/Source/hive-release-HDP-2.5.0.0-tag/shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java:[1423,12] cannot find symbol
[ERROR] symbol:   variable CallerContext
[ERROR] location: class org.apache.hadoop.hive.shims.Hadoop23Shims
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hive-shims-0.23

Re: Failed to compile HDP hive 2.5.0.0 and 2.4.2.40 with the same error.

rajkumar_singh — Sat, 10 Dec 2016 12:38:37 GMT

@Lawrence Lau could you please try building hive after adding hortonworks repo in you pom

 <repositories>    
<repository> <id>public</id> <url>http://nexus-private.hortonworks.com/nexus/content/groups/public

</url> </repository> </repositories>

Re: Failed to compile HDP hive 2.5.0.0 and 2.4.2.40 with the same error.

onlyforcloud — Sun, 11 Dec 2016 18:42:56 GMT

Thanks very much for your help. I tried your way, it works.

I added those below into my pom.

<code>

http://repo.hortonworks.com/content/repositories/releases/

http://repo.hortonworks.com/content/repositories/jetty-hadoop/

http://nexus-private.hortonworks.com/nexus/content/groups/public

</code>

This time when I compiled hive, a new error occurred. Later if I solved it, I'll give my way here.

Re: Failed to compile HDP hive 2.5.0.0 and 2.4.2.40 with the same error.

onlyforcloud — Tue, 20 Dec 2016 10:10:02 GMT

I simply compiled hdp hive 2.5.0.0 by "mvn clean install -Dmaven.test.skip=true".

After that I got this error.

Later I changed into "mvn clean install -Dmaven.test.skip=true -Phadoop-2 " and everything is ok.

question Re: Failed to compile HDP hive 2.5.0.0 and 2.4.2.40 with the same error. in Archives of Support Questions (Read Only)

Failed to compile HDP hive 2.5.0.0 and 2.4.2.40 with the same error.

Re: Failed to compile HDP hive 2.5.0.0 and 2.4.2.40 with the same error.

Re: Failed to compile HDP hive 2.5.0.0 and 2.4.2.40 with the same error.

Re: Failed to compile HDP hive 2.5.0.0 and 2.4.2.40 with the same error.