@Ivan Majnaric Nothing should happen. Your spark application will continue running and you will be still be able to submit applications. You will loose access to the spark history ui of course. HTH *** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer. ... View more

@Michael Bronson Perhaps you can use services rest api to check like this: curl -iv -u admin -X GET http://AMBARI_HOST:8080/api/v1/clusters/CLUSTER_NAME/services/KAFKA If service is not installed you will received a response indicating resource does not exist. HTH *** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer. ... View more

@Anuj Tanwar AFAIK updates are supported with spark jdbc. Alternative is to use standard jdbc - You can read more here https://stackoverflow.com/questions/34643200/spark-dataframes-upsert-to-postgres-table *** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer. ... View more

@John Doe Could you try running on yarn client mode instead of local? I think this will help resolving the problem you have now. $ pyspark --master yarn from pyspark.sql import SparkSession spark =SparkSession.builder.appName("test").enableHiveSupport().getOrCreate() spark.sql("show databases").show() spark.sql("create database if not exists NEW_DB") Note: If you comment this post make sure you tag my name. And If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer. HTH ... View more