@Anpan K Usually upon a saved change/new service install ambari will ask you to restart one or more services. . Ambari is aware that certain actions performed on service Y will require a restart of service X - Hence marks as restart is required. . Why restart is required sometimes requires you to know what have been latest changes performed. Check the services version and see if any changes have been done recently. Or perhaps a new service was added. HTH ... View more

@JAy PaTel Try with putty alternative pscp, hopefully this will work. Or else you can create a shared folder on your virtualbox and use it to move files from windows to virtual machine. ... View more

@JAy PaTel try with 127.0.0.1 instead of localhost. scp -v -p 2222 datafile.txt root@127.0.0.1:/ HTH *** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer. ... View more

@JAy PaTel The above means that during startup of the spark application the sparkcontext was not initialized. I've seen this particular error in cases where application code is doing something previous to creating the sparkcontext/sparksession and this pre-creation of sparkcontext code piece is delayed (for any reasons) leading to this issue. I recommend you review your code with detail. Take oozie out of the picture. Also try run it in yarn-cluster and/or yarn-client mode, perhaps it will fail as well and this will simplify the troubleshooting. HTH ... View more

@JAy PaTel The error message you see is very generic. When dealing with this type of errors if possible you could set yarn.nodemanager.delete.debug-delay-sec=600 this will give you some time to go to the actual node where this is failing and dig into the yarn local dir to hopefully find the actual cause for the job to fail. Check under /hadoop/yarn/local/usercache for the application id and any log files that could potentially lead to a better understanding of the problem. HTH ... View more

@priyal patel First make sure you know if OOM is happening on driver or in executor. You can find this by looking at the logs. To test I suggest you increase the --driver-memory to 10g or even 20g and see what happens. Also try running on yarn-client mode instead of yarn-cluster. If OOM error comes on the sdtout of spark-submit you will know the driver is running out of memory. Else you can check the yarn logs -applicationId <appId> to see what happened on the executor side. HTH *** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer. ... View more

@RAUI 1. Currently i am writing dataframe into hive table using insertInto() and mode("append") i am able to write the data into hive table but i am not sure that is the correct way to do it? Please review the following link, I'm hoping it helps address this question: https://stackoverflow.com/questions/47844808/what-are-the-differences-between-saveastable-and-insertinto-in-different-savemod 2. For the exception I would suggest you open separate thread and add more information - including full error stack, spark client command line arguments and code you are running that is failing. HTH ... View more