Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi Processor multi query using incoming connection


NiFi Processor multi query using incoming connection



We need to develop a NiFi dataflow that have a lot of sources that come to the cluster by different ways, most of them by querying different tables of different databases. For simplicity drawing the dataflow, we're thinking in some processor (or combination of) that we can query different tables using data coming from a flowfile (that includes information about de database controller).

Is it possible to do it with the current processors?

Do you think it's possible to develop a custom processor that it has this characteristics?



Re: NiFi Processor multi query using incoming connection

Currently the database related processors require selecting an instance of DBCPConnectionPool which is tied to a single database. You can make the table name and sql dynamic, but not the actual database because you have to select the DBCPConnectionPool ahead of time.

There have been some recent improvements in master that should make it possible to implement something like this:

Don't have an account?
Coming from Hortonworks? Activate your account here