We need to develop a NiFi dataflow that have a lot of sources that come to the cluster by different ways, most of them by querying different tables of different databases. For simplicity drawing the dataflow, we're thinking in some processor (or combination of) that we can query different tables using data coming from a flowfile (that includes information about de database controller).
Is it possible to do it with the current processors?
Do you think it's possible to develop a custom processor that it has this characteristics?
Currently the database related processors require selecting an instance of DBCPConnectionPool which is tied to a single database. You can make the table name and sql dynamic, but not the actual database because you have to select the DBCPConnectionPool ahead of time.
There have been some recent improvements in master that should make it possible to implement something like this: