Hi @Meepoljd Have you checked any exceptions/warnings in the spark application logs and event logs. And also check you listeners are implemented properly and spark application handled exceptions correctly.
... View more
Hi @SAMSAL I think you want to run the spark application using Standalone mode. Please follow the following steps: 1. Install the Apache Spark 2. Start the Standalone master and workers. By default master will start with port 7777. Try to access and Standalone UI and see all workers are running expected. 3. Once it is running as expected then submit spark application by specifying standalone master host with 7777
... View more
Have you tried to restart the cluster and cluster services. I think you need to keep all services are running because either directly or indirectly one service is dependent on other service.
... View more
I think you need to verify the yarn and spark resources are configured properly. If yes then go and check from spark ui, it will show driver memory and executor memory. It is coming as expected then safely you can ignore it.
... View more
Hi @adhishankarit Spark applications will run using spark engine and not a tez engine unlike hive. You no need to set any engine from spark side. If you want to run hive queries then you can set engines like Tez, Spark, MR
... View more
I think you don't have sufficient resources to run the job for queue root.hdfs. Verify is there any pending running jobs/application in the root.hdfs queue from Resource Manager UI. If it is running kill those if it is not required. And also verify from spark side you have given less resource to test it.
... View more
