How to display more then 1000 rows in Hue using Spark SQL Livy Interpreter

Question

Dear FriendsThe concerned Cloudera Services are: Spark 3 , Livy for Spark 3, HueAfter I have added Spark SQL interpreter in Hue I am struggling  to get more than 1000 rows results once querying  the DB. Do you know what parameter should be used to change the default quantity. I Already tried to use :spark.livy.sql.max-resultCM --> Livy for Spark3 --> Search for "Livy for Spark 3 Client Advanced Configuration Snippet (Safety Valve) for livy3-conf/livy-client.conf" and add the above parameter with value.spark.livy.sql.max-result=2000However after restarting the concerns services - nothing has changed.  Hereunder you can see what is all about:I would like to change this limitation to e.g. 5000. But not sure on what mentioned services this should be adjusted on.I really appreciate your help in this.Thank you

RangaReddy · Accepted Answer

Hi @Bartlomiej You can use livy.rsc.sql.num-rows parameter to adjust the number of rows wants to display.  CM --> Livy for Spark3 --> Search for "Livy Server for Spark 3 Advanced Configuration Snippet (Safety Valve) for livy3-conf/livy.conf" and add the above parameter with value.livy.rsc.sql.num-rows=5000   I have created a test_3000 table by inserting 1 to 2999 records. While running the query from Hue Spark3 editor you can see the following output:

Bartlomiej · Answer

Yes, this is it. Thank you so much for the prompt response.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.