@rangareddyy   I recently did something similar, and each time, in the code, that i wanted to send flowfile i used:   session.transfer(flowFile2, REL_SUCCESS) session.commit()  It is my understanding in a single flowfile example, the commit() is inferred upon script completion. ... View more

@cnelson2 This is legit!  Excellent work my friend!! ... View more

@SandyClouds Since CaptureChangeMysql does not accept incoming flowfiles you would have to create a separate CaptureChangeMysql flow for each database.  This would be exhaustive on 100s of manually created flows.  I would have to automate the creation of these flows in a normal nifi environment.   In some of the CDC demos I am working with now,  i am using CDP Public Cloud Data Flow and able to paramaterize my flows and create flow definitions that are very dynamic then deploy them without touching NiFi or NiFi API and only changing the parameters.   This would be an interesting approach to solving 100s of data flows.      Another approach I use to begin CDC conversations without actually using "CDC" is to use ExecuteSQL which does accept incoming flowfiles.  I use this to grab an entire database and all tables, put all the data in kafka, and all the schema in Schema Registry.  This enables me to start to build deeper CDC logic around the same data, including using the same kafka topic per table for updates from other systems, then ultimately deciding how to handle deletes.     That said, the complications in creating and operating these types of flows are only enabled when you begin to start capturing data.  To me CDC is a journey, not a destination, and we can only begin down that path when we start putting the above concepts in place whether that is CaptureChangeMysql or ExecuteSql with deeper logic for handling changes of the source database.   ... View more

@AbhishekSingh I believe you can accomplish this with some very complicated Expression Language chaining.  It is possible to have many Expression Languages combined into one expression.   Reference:   https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html   I do not have time to prove this out, but in theory,  you would combine lesson learned in your other post (expression language to change the database.table with Replace, ifElse and maybe Equals to get your desired outcome. ... View more

@araujo It would be awesome if you could also link a flow definition file....  ... View more

I have done similar here when I need to deliver jar files to all nodes.   It's really a "this is not how things are done", but in this case I did not have access to the node's file system without doing this in a flow.  So that said, it works great!     The first proc creates a flowfile on all nodes (even when I dont know the number), then it checks, if not found, proceeds to get the file and write it to the file system.            ... View more

Fun with python,  you are going to need to resolve all dependencies.  I am not familiar with the last error, but its definitely saying psycopg2 is not found..  ... View more

Yes, not available before 1.16.  Definitely a great new feature!! ... View more