Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

spark3 for cdp error

avatar
Contributor

sql("show databases").show()

dfdf_5-1655285019756.png

 

sql("show tables").show()

error

dfdf_0-1655284896927.pngdfdf_1-1655284917055.pngdfdf_2-1655284939645.pngdfdf_3-1655284956902.pngdfdf_4-1655284973345.png

 

1 ACCEPTED SOLUTION

avatar
Master Collaborator

Hi @dfdf, I tried in my cluster with both Spark2 and Spark3 on the same version which you tried but I can able to get the results without any issues. 

Spark2: 2.4.7.7.1.7.1000-141

jagadeesan_0-1656455046113.png

Spark3 : 3.2.1.3.2.7171000.1-1

jagadeesan_1-1656455079072.png

 

Are you still seeing this issue? Please can you share the reproduce steps that I can try from my side to reproduce this issue in my cluster? Thanks  

View solution in original post

5 REPLIES 5

avatar
Master Collaborator

Hi @dfdf, I am not able to reproduce this issue. I can able to get the table details while running the queries in the Spark3 session. Please can you help us with the exact version of Spark3 and Hive version that you are running on your environment? For example, you can get the value by running spark3-shell --version

Please verify are you seeing any errors or alerts related to the Hive service? Also, can you try to run similar queries directly from Hive and see whether are you getting the results? 

avatar
Contributor

cm:7.3.1

cdp:7.1.6

sql:

spark2 is ok!

dfdf_0-1655343623842.png

spark3 is error

dfdf_1-1655344546388.png

 

avatar
Contributor

dfdf_2-1655344783098.png

 

avatar
Master Collaborator

Hi @dfdf, I tried in my cluster with both Spark2 and Spark3 on the same version which you tried but I can able to get the results without any issues. 

Spark2: 2.4.7.7.1.7.1000-141

jagadeesan_0-1656455046113.png

Spark3 : 3.2.1.3.2.7171000.1-1

jagadeesan_1-1656455079072.png

 

Are you still seeing this issue? Please can you share the reproduce steps that I can try from my side to reproduce this issue in my cluster? Thanks  

avatar
Community Manager

@dfdf, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  If you are still experiencing the issue, can you provide the information @jagadeesan has requested?



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: