Check out this article:   https://community.cloudera.com/t5/Community-Articles/NiFi-Sizing-Guide-Deployment-Best-Practices/ta-p/246781     ... View more

@Nifi_Al Its hard to guess here without seeing your flow, but i think here are a few ideas to resolve:   Use GenerateFlowFIle with content of the json you want to send to PutElasticsearchHttp Use AttributesToJson to write attribute to content of flowfile, then send that to PutElasticsearchHttp   To test basic connectivity, i would use the first method, then build a working flow from there.  I always work in small operational samples when trying to figure out how to configure a certain processor. ... View more

I think you will also need to do the transfer/commit in the each list logic ... View more

@rangareddyy You will need to add attributes to the flowfile: flowFile = session.putAttribute(flowFile, 'myAttr', 'myValue') Reference:   https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-1/ta-p/248922 ... View more

This is a groovy error.  I would suggest having a working groovy script executeable outside of nifi before trying to modify and execute that code in the context of ExecuteScript. ... View more

@rangareddyy   I recently did something similar, and each time, in the code, that i wanted to send flowfile i used:   session.transfer(flowFile2, REL_SUCCESS) session.commit()  It is my understanding in a single flowfile example, the commit() is inferred upon script completion. ... View more

@cnelson2 This is legit!  Excellent work my friend!! ... View more

@SandyClouds Since CaptureChangeMysql does not accept incoming flowfiles you would have to create a separate CaptureChangeMysql flow for each database.  This would be exhaustive on 100s of manually created flows.  I would have to automate the creation of these flows in a normal nifi environment.   In some of the CDC demos I am working with now,  i am using CDP Public Cloud Data Flow and able to paramaterize my flows and create flow definitions that are very dynamic then deploy them without touching NiFi or NiFi API and only changing the parameters.   This would be an interesting approach to solving 100s of data flows.      Another approach I use to begin CDC conversations without actually using "CDC" is to use ExecuteSQL which does accept incoming flowfiles.  I use this to grab an entire database and all tables, put all the data in kafka, and all the schema in Schema Registry.  This enables me to start to build deeper CDC logic around the same data, including using the same kafka topic per table for updates from other systems, then ultimately deciding how to handle deletes.     That said, the complications in creating and operating these types of flows are only enabled when you begin to start capturing data.  To me CDC is a journey, not a destination, and we can only begin down that path when we start putting the above concepts in place whether that is CaptureChangeMysql or ExecuteSql with deeper logic for handling changes of the source database.   ... View more

@AbhishekSingh I believe you can accomplish this with some very complicated Expression Language chaining.  It is possible to have many Expression Languages combined into one expression.   Reference:   https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html   I do not have time to prove this out, but in theory,  you would combine lesson learned in your other post (expression language to change the database.table with Replace, ifElse and maybe Equals to get your desired outcome. ... View more