org.apache.log4j.LogManager.getRootLogger().info("This is an information log");
I am abit confused as there is no yarn installed on the cluster and I can not see logs in Spark history job when I go to executors. Can anyone suggest me how to get my own logs inside Spark applications. when running by Livyclient.
I changed the mode of application to run on yarn-cluster mode and now I get the logs..but I don't see any yarn on our cluster installed. I cant run yarn commands on cluster..but can run Spark jobs on yarn cluster mode using livy..I need to run yarn commands as well