Created 06-16-2016 08:23 PM
How will Apache Calcite be used to enhance Apache NiFi's SQL abilities?
Created 06-16-2016 09:07 PM
there is a JIRA created for bringing Calcite into a NiFi processor - https://issues.apache.org/jira/browse/NIFI-1280. The idea originally was to use it to filter out specific columns in incoming CSV data. However, as we looked at it, we found that there is a lot more that this can do! Initially, it will likely be used to simply run SQL over CSV data, with each incoming FlowFile being transformed into an outgoing FlowFile. Eventually, I would like to see additional data formats being introduced into this, so that SQL could be run over any number of different data formats to filter, transform, etc.
Created 06-16-2016 09:07 PM
there is a JIRA created for bringing Calcite into a NiFi processor - https://issues.apache.org/jira/browse/NIFI-1280. The idea originally was to use it to filter out specific columns in incoming CSV data. However, as we looked at it, we found that there is a lot more that this can do! Initially, it will likely be used to simply run SQL over CSV data, with each incoming FlowFile being transformed into an outgoing FlowFile. Eventually, I would like to see additional data formats being introduced into this, so that SQL could be run over any number of different data formats to filter, transform, etc.