Options
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
Guru
Created on 02-27-2018 10:20 PM
Building Apache Tez with Apache Hadoop 2.8.0 or later fails due to client/server jar separation in Hadoop [1]. The build fails with the following error:
[ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /src/tez/tez-api/src/test/java/org/apache/tez/client/TestTezClientUtils.java:[48,30] cannot find symbol symbol: class DistributedFileSystem location: package org.apache.hadoop.hdfs [ERROR] /src/tez/tez-api/src/test/java/org/apache/tez/client/TestTezClientUtils.java:[680,50] cannot find symbol symbol: class DistributedFileSystem location: class org.apache.tez.client.TestTezClientUtils [ERROR] /src/tez/ecosystem/tez/tez-api/src/test/java/org/apache/tez/common/TestTezCommonUtils.java:[62,42] cannot access org.apache.hadoop.hdfs.DistributedFileSystem
To get Tez to compile successfully, you will need to use the new hadoop28 profile introduced by TEZ-3690 [2]. E.g. here is how you compile Tez with Apache Hadoop 3.0.0:
mvn clean package -DskipTests=true -Dmaven.javadoc.skip=true -Phadoop28 -Dhadoop.version=3.0.0
References:
1. HDFS-6200: Create a separate jar for hdfs-client
2. TEZ-3690: Tez on hadoop 3 build failed due to hdfs client/server jar separation.
2,251 Views