https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-a-good-idea-generally-to-use-NiFi-as-a-scheduler-for/m-p/118181#M42929 <P>Short answer - kinda, it depends on your expectations of a scheduler. NiFi is perfectly capable of kicking off jobs once it prepares and lands the data. The nature of a scheduler, though, is to often wait for a job to finish, retry, act on it, etc. Depending on the actual infrastructure, you may find NiFi less convenient to handle such hierarchical dependencies than a scheduler that was designed for this purpose.</P> Fri, 07 Oct 2016 01:49:03 GMT andrewg 2016-10-07T01:49:03Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-a-good-idea-generally-to-use-NiFi-as-a-scheduler-for/m-p/118180#M42928 <P>For example, if NiFi delivers data to HDFS somehow, and I have a sequence of Hive and/or Spark jobs that need to run against that data in an HDP cluster, is it a good idea to orchestrate those successive Hive/Spark jobs using the NiFi executeProcess/excecuteScript processors? In contrast to writing Oozie workflows and Falcon processes.</P> Fri, 07 Oct 2016 01:18:41 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-a-good-idea-generally-to-use-NiFi-as-a-scheduler-for/m-p/118180#M42928 mark_poscablo 2016-10-07T01:18:41Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-a-good-idea-generally-to-use-NiFi-as-a-scheduler-for/m-p/118181#M42929 <P>Short answer - kinda, it depends on your expectations of a scheduler. NiFi is perfectly capable of kicking off jobs once it prepares and lands the data. The nature of a scheduler, though, is to often wait for a job to finish, retry, act on it, etc. Depending on the actual infrastructure, you may find NiFi less convenient to handle such hierarchical dependencies than a scheduler that was designed for this purpose.</P> Fri, 07 Oct 2016 01:49:03 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-a-good-idea-generally-to-use-NiFi-as-a-scheduler-for/m-p/118181#M42929 andrewg 2016-10-07T01:49:03Z https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-a-good-idea-generally-to-use-NiFi-as-a-scheduler-for/m-p/118182#M42930 <P>Use Falcon for Map Reduce, Sqoop and Flume jobs.</P><P>Use NiFi for everything else.</P> Fri, 07 Oct 2016 01:53:01 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/Is-it-a-good-idea-generally-to-use-NiFi-as-a-scheduler-for/m-p/118182#M42930 TimothySpann 2016-10-07T01:53:01Z