Support Questions
Find answers, ask questions, and share your expertise

How will Apache Calcite be integrated into Apache NiFi?

Super Guru

How will Apache Calcite be used to enhance Apache NiFi's SQL abilities?

1 ACCEPTED SOLUTION

Rising Star
@Timothy Spann

there is a JIRA created for bringing Calcite into a NiFi processor - https://issues.apache.org/jira/browse/NIFI-1280. The idea originally was to use it to filter out specific columns in incoming CSV data. However, as we looked at it, we found that there is a lot more that this can do! Initially, it will likely be used to simply run SQL over CSV data, with each incoming FlowFile being transformed into an outgoing FlowFile. Eventually, I would like to see additional data formats being introduced into this, so that SQL could be run over any number of different data formats to filter, transform, etc.

View solution in original post

1 REPLY 1

Rising Star
@Timothy Spann

there is a JIRA created for bringing Calcite into a NiFi processor - https://issues.apache.org/jira/browse/NIFI-1280. The idea originally was to use it to filter out specific columns in incoming CSV data. However, as we looked at it, we found that there is a lot more that this can do! Initially, it will likely be used to simply run SQL over CSV data, with each incoming FlowFile being transformed into an outgoing FlowFile. Eventually, I would like to see additional data formats being introduced into this, so that SQL could be run over any number of different data formats to filter, transform, etc.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.