Skip to main content
dfdf
dfdf
Silver ⭐️⭐️⭐️
June 15, 2022
Solved

spark3 for cdp error

sql("show databases").show()

dfdf_5-1655285019756.png

 

sql("show tables").show()

error

dfdf_0-1655284896927.pngdfdf_1-1655284917055.pngdfdf_2-1655284939645.pngdfdf_3-1655284956902.pngdfdf_4-1655284973345.png

 

jagadeesan
jagadeesanAnswer
Platinum ⭐️
June 28, 2022

Hi @dfdf, I tried in my cluster with both Spark2 and Spark3 on the same version which you tried but I can able to get the results without any issues. 

Spark2: 2.4.7.7.1.7.1000-141

jagadeesan_0-1656455046113.png

Spark3 : 3.2.1.3.2.7171000.1-1

jagadeesan_1-1656455079072.png

 

Are you still seeing this issue? Please can you share the reproduce steps that I can try from my side to reproduce this issue in my cluster? Thanks  

View in thread

5 replies

jagadeesan
Platinum ⭐️
jagadeesan
Platinum ⭐️
June 15, 2022

Hi @dfdf, I am not able to reproduce this issue. I can able to get the table details while running the queries in the Spark3 session. Please can you help us with the exact version of Spark3 and Hive version that you are running on your environment? For example, you can get the value by running spark3-shell --version

Please verify are you seeing any errors or alerts related to the Hive service? Also, can you try to run similar queries directly from Hive and see whether are you getting the results? 

Translate
    dfdf
    dfdfAuthor
    Silver ⭐️⭐️⭐️
    June 16, 2022

    cm:7.3.1

    cdp:7.1.6

    sql:

    spark2 is ok!

    dfdf_0-1655343623842.png

    spark3 is error

    dfdf_1-1655344546388.png

     

    Translate
    dfdf
    dfdfAuthor
    Silver ⭐️⭐️⭐️
    June 16, 2022

    dfdf_2-1655344783098.png

     

    Translate
    jagadeesan
    Platinum ⭐️
    jagadeesanAnswer
    Platinum ⭐️
    June 28, 2022

    Hi @dfdf, I tried in my cluster with both Spark2 and Spark3 on the same version which you tried but I can able to get the results without any issues. 

    Spark2: 2.4.7.7.1.7.1000-141

    jagadeesan_0-1656455046113.png

    Spark3 : 3.2.1.3.2.7171000.1-1

    jagadeesan_1-1656455079072.png

     

    Are you still seeing this issue? Please can you share the reproduce steps that I can try from my side to reproduce this issue in my cluster? Thanks  

    Translate
    VidyaSargur
    Community Manager
    VidyaSargur
    Community Manager
    July 4, 2022

    @dfdf, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  If you are still experiencing the issue, can you provide the information @jagadeesan has requested?

    Translate
    Regards, Vidya Sargur, Community Manager Was your question answered? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
    Terms & ConditionsAccessibility statement