02-06-2018 07:49 PM - last edited on 02-07-2018 05:57 AM by cjervis
I am in dilemma about which option is best while scheduling Hadoop workflows.
We have Tidal as enterprise scheduler without native Hadoop plugins.
We are ingesting files to Hadoop from Edge node using HDFS put command wrapped in shell scripts and these shell scripts scheduled using Tidal.
Is it a good practice to create shell scripts for rest of jobs those are having Hive, Sqoop, Impala, Spark actions and run it on Edge node scheduling in Tidal OR schedule all these actions in Oozie ?
What are pros and cons of both the options ?