Unable to run Query on Hive-3.0 with ADLS

[#1]

18/11/12 10:37:25 [main]: INFO jdbc.HiveConnection: Connected to server1.test.com:10000 Connected to: Apache Hive (version 3.0.0.3.0.0.0-1334) Driver: Hive JDBC (version 3.0.0.3.0.0.0-1334) Transaction isolation: TRANSACTION_REPEATABLE_READ Beeline version 3.0.0.3.0.0.0-1334 by Apache Hive 0: jdbc:hive2://server1.test.com:> select count(*) from lnd_edw4x_qn1useh1.redemption_fact_t;

INFO : Compiling command(queryId=hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216): select count(*) from lnd_edw4x_qn1useh1.redemption_fact_t

INFO : Semantic Analysis Completed

INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null) INFO : Completed compiling command(queryId=hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216); Time taken: 1.218 seconds

INFO : Executing command(queryId=hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216): select count(*) from lnd_edw4x_qn1useh1.redemption_fact_t

INFO : Query ID = hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216 INFO : Total jobs = 1 INFO : Launching Job 1 out of 1 INFO : Starting task [Stage-1:MAPRED] in serial mode

INFO : Subscribed to counters: [] for queryId: hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216

INFO : Tez session hasn't been created yet. Opening session

INFO : Dag name: select count(*) from ...h1.redemption_fact_t (Stage-1)

ERROR : Status: Failed ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1542008498326_0022_1_00, diagnostics=[Vertex vertex_1542008498326_0022_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: redemption_fact_t initializer failed, vertex=vertex_1542008498326_0022_1_00 [Map 1], java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.adl.AdlFileSystem not found

at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2595)

at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3320)

I am already having below jars in Hadoop/hive libs still it's not working:

hadoop-azure-3.0.0.3.0.0.0-1334.jar, hadoop-azure-datalake-3.0.0.3.0.0.0-1334.jar, azure-data-lake-store-sdk-2.2.7.jar

[#2]

Hi @arjun more

Maybe that the below jars are not there for the yarn containers for it to make use of when hive launches the job. You can try adding in these jars with 'add jar <path_to_jar>/jarfilename.jar; and see if this helps.

[#3]

What version of HDP are you using ?

[#4 · Accepted solution]

@Prabhu M @Mahesh Balakrishnan : Issue has been resolved.

It's a bug with Tez version 0.9.0 (HDP-3.0.0.0) which they have fixed in Tez 0.9.1 version(HDP-3.0.1.0)