Hello Experts, After going through Livy documentations available on the internet. It looks like Livy can impersonate the end user, we decide to test it for our use case before setting it up in Production. I could run simple wordcount problem through Livy interpreter and read and write data to HDFS. Next hurdle is to use it run queries, listing db/tables,etc. I am trying to run using (one at a time, but I see the same error) using zeppelin notebook in a kerberized environment. %livy %livy.spark %livy.sql sqlContext.sql("show databases") // Doesn't work. show databases // does not work. sqlContext.sql("show databases").show() //does not work Every time I see : java.lang.RuntimeException: [1.1] failure: ``with'' expected but identifier "use or select or sqlContext" found. Any idea if I am missing something here? Sc However, when I try to run following I see the list of tables (but can't list databases yet) 1) Read the data from HDFS using sc.textFile() 2) Define Case class 3) Parse the file from step#1, and build the RDD of case objects 4) convert to DataFrame and create temp table 5) sqlContext.sql("show tables").toDF().show() // This lists only temp table. I am interested in listing ALL THE DATBASES and ALL THE TABLES ( TEMP and PERM) What is that I am missing here? Thanks
... View more
