How will Apache Calcite be integrated into Apache NiFi?

[#1]

How will Apache Calcite be used to enhance Apache NiFi's SQL abilities?

[#2 · Accepted solution]

@Timothy Spann

there is a JIRA created for bringing Calcite into a NiFi processor - https://issues.apache.org/jira/browse/NIFI-1280. The idea originally was to use it to filter out specific columns in incoming CSV data. However, as we looked at it, we found that there is a lot more that this can do! Initially, it will likely be used to simply run SQL over CSV data, with each incoming FlowFile being transformed into an outgoing FlowFile. Eventually, I would like to see additional data formats being introduced into this, so that SQL could be run over any number of different data formats to filter, transform, etc.