Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to run Query on Hive-3.0 with ADLS

Unable to run Query on Hive-3.0 with ADLS

New Contributor

18/11/12 10:37:25 [main]: INFO jdbc.HiveConnection: Connected to server1.test.com:10000 Connected to: Apache Hive (version 3.0.0.3.0.0.0-1334) Driver: Hive JDBC (version 3.0.0.3.0.0.0-1334) Transaction isolation: TRANSACTION_REPEATABLE_READ Beeline version 3.0.0.3.0.0.0-1334 by Apache Hive 0: jdbc:hive2://server1.test.com:> select count(*) from lnd_edw4x_qn1useh1.redemption_fact_t;

INFO : Compiling command(queryId=hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216): select count(*) from lnd_edw4x_qn1useh1.redemption_fact_t

INFO : Semantic Analysis Completed

INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null) INFO : Completed compiling command(queryId=hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216); Time taken: 1.218 seconds

INFO : Executing command(queryId=hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216): select count(*) from lnd_edw4x_qn1useh1.redemption_fact_t

INFO : Query ID = hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216 INFO : Total jobs = 1 INFO : Launching Job 1 out of 1 INFO : Starting task [Stage-1:MAPRED] in serial mode

INFO : Subscribed to counters: [] for queryId: hive_20181112103728_c02d0353-978c-457c-bde7-9ae45fcb1216

INFO : Tez session hasn't been created yet. Opening session

INFO : Dag name: select count(*) from ...h1.redemption_fact_t (Stage-1)

ERROR : Status: Failed ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1542008498326_0022_1_00, diagnostics=[Vertex vertex_1542008498326_0022_1_00 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: redemption_fact_t initializer failed, vertex=vertex_1542008498326_0022_1_00 [Map 1], java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.adl.AdlFileSystem not found

at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2595)

at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3320)

I am already having below jars in Hadoop/hive libs still it's not working:

hadoop-azure-3.0.0.3.0.0.0-1334.jar, hadoop-azure-datalake-3.0.0.3.0.0.0-1334.jar, azure-data-lake-store-sdk-2.2.7.jar

3 REPLIES 3

Re: Unable to run Query on Hive-3.0 with ADLS

Expert Contributor

Hi @arjun more

Maybe that the below jars are not there for the yarn containers for it to make use of when hive launches the job. You can try adding in these jars with 'add jar <path_to_jar>/jarfilename.jar; and see if this helps.

Highlighted

Re: Unable to run Query on Hive-3.0 with ADLS

Contributor

What version of HDP are you using ?

Re: Unable to run Query on Hive-3.0 with ADLS

New Contributor

@Prabhu M @Mahesh Balakrishnan : Issue has been resolved.

It's a bug with Tez version 0.9.0 (HDP-3.0.0.0) which they have fixed in Tez 0.9.1 version(HDP-3.0.1.0)

93714-aa.jpg