Hello all,
I am creating a processor group that read from kafka topic and write it to the clickhouse database. I am using stateless mechanism to ensure that when there is a problem during execution, nifi crash, or nifi restarted or clickhouse database return error, kafka offset will not be committed and process will be retry.
![rtambun_0-1708666355928.png rtambun_0-1708666355928.png](https://community.cloudera.com/t5/image/serverpage/image-id/39765i64EE19B0A4A11DFC/image-size/medium?v=v2&px=400)
Unfortunately clickhouse will create a new row for duplicated message. In order to avoid duplicate message, i would like to check first the database and see if i have duplicated message before processing. Have someone create similiar use case as this one?