NiFI Create multiple jobs

[#1]

Hi Guys,

We are having multiple jobs in Informatica;

Job1: X application files load to Y DB

Job2: A application files load to Y DB

Job3: Y DB table load to Z DB

I want to create 3 jobs for 3 different needs; how do i achieve it using NIFI?

Regards,

Navin

[#2]

It sounds like you are trying to convert a job-based batch use case into a real-time streaming use case. You will have to re-think what your requirements and goals are for your process, as NiFi doesn't operate in jobs but in streaming data that is constantly coming in.

In NiFi, you may decide to implement your workflow the following way. Set your processor to monitor an HDFS directory for new files and load them into "Y DB" whenever new files are found with a scheduling interval of 1 minute (or 5, or 10). You would use the ListFiles processor to monitor for new files in a directory. That would accomplish your first 2 jobs. For the third job, you could use the QueryDatabaseTable to scan for changes every so often (say, 1 minute) and load those changes into "Z DB". This would move your processing from a batch-oriented nature to a real-time streaming workflow.