question Re: spark3 for cdp error in Support Questions

spark3 for cdp error

dfdf — Wed, 15 Jun 2022 09:23:57 GMT

sql("show databases").show()

sql("show tables").show()

error

Re: spark3 for cdp error

jagadeesan — Wed, 15 Jun 2022 16:40:22 GMT

Hi @dfdf, I am not able to reproduce this issue. I can able to get the table details while running the queries in the Spark3 session. Please can you help us with the exact version of Spark3 and Hive version that you are running on your environment? For example, you can get the value by running spark3-shell --version

Please verify are you seeing any errors or alerts related to the Hive service? Also, can you try to run similar queries directly from Hive and see whether are you getting the results?

Re: spark3 for cdp error

dfdf — Thu, 16 Jun 2022 01:55:57 GMT

cm:7.3.1

cdp:7.1.6

sql:

spark2 is ok!

spark3 is error

Re: spark3 for cdp error

dfdf — Thu, 16 Jun 2022 01:59:50 GMT

Re: spark3 for cdp error

jagadeesan — Tue, 28 Jun 2022 22:31:34 GMT

Hi @dfdf, I tried in my cluster with both Spark2 and Spark3 on the same version which you tried but I can able to get the results without any issues.

Spark2: 2.4.7.7.1.7.1000-141

Spark3 : 3.2.1.3.2.7171000.1-1

Are you still seeing this issue? Please can you share the reproduce steps that I can try from my side to reproduce this issue in my cluster? Thanks

Re: spark3 for cdp error

VidyaSargur — Mon, 04 Jul 2022 05:22:10 GMT

@dfdf, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. If you are still experiencing the issue, can you provide the information @jagadeesan has requested?