https://community.cloudera.com/t5/Support-Questions/How-to-consume-kafka-messages-from-a-specific-offset-in-nifi/m-p/387017#M246220 <P>Not possible, there is a reason for that&nbsp;<SPAN>if we allow to add a property to set an initial offset, every time that processor was stopped and started we would again start at that configured offset,&nbsp;</SPAN><SPAN>with console consumer, you only pass the initial offset on the first execution</SPAN></P><DIV class="c-virtual_list__item"><DIV class="c-message_kit__background c-message_kit__background--hovered p-message_pane_message__message c-message_kit__message"><DIV class="c-message_kit__hover c-message_kit__hover--hovered"><DIV class="c-message_kit__actions c-message_kit__actions--above"><DIV class="c-message_kit__gutter"><DIV class="c-message_kit__gutter__right"><DIV class="c-message_kit__blocks c-message_kit__blocks--rich_text"><DIV class="c-message__message_blocks c-message__message_blocks--rich_text"><DIV class="p-block_kit_renderer"><DIV class="p-block_kit_renderer__block_wrapper p-block_kit_renderer__block_wrapper--first"><DIV class="p-rich_text_block"><DIV class="p-rich_text_section">then subsequent executions use the last stored offset in Kafka, which is not the case here with the processor.</DIV><DIV class="p-rich_text_section">Thus only two options are&nbsp;<SPAN>"earliest" or "latest".</SPAN></DIV><DIV class="p-rich_text_section">&nbsp;</DIV><DIV class="p-rich_text_section"><SPAN>Thank you </SPAN></DIV><DIV class="p-rich_text_section">&nbsp;</DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV> Tue, 23 Apr 2024 08:46:04 GMT ckumar 2024-04-23T08:46:04Z https://community.cloudera.com/t5/Support-Questions/How-to-consume-kafka-messages-from-a-specific-offset-in-nifi/m-p/386960#M246197 <P>Hello Experts,</P><P>I am using Apache Nifi 1.25 now and have a requirement to consume data from a specific offset of a topic partition.</P><P>I see in Nifi ConsumeKafka processor to configure only "earliest" or "latest" but not specific offset number, is there any way to achieve this ?</P><P>&nbsp;</P><P>Any info/suggestion would be appreciated.</P><P>&nbsp;</P><P>Thanks,</P><P>Mahendra</P> Mon, 22 Apr 2024 17:05:43 GMT https://community.cloudera.com/t5/Support-Questions/How-to-consume-kafka-messages-from-a-specific-offset-in-nifi/m-p/386960#M246197 hegdemahendra 2024-04-22T17:05:43Z https://community.cloudera.com/t5/Support-Questions/How-to-consume-kafka-messages-from-a-specific-offset-in-nifi/m-p/387017#M246220 <P>Not possible, there is a reason for that&nbsp;<SPAN>if we allow to add a property to set an initial offset, every time that processor was stopped and started we would again start at that configured offset,&nbsp;</SPAN><SPAN>with console consumer, you only pass the initial offset on the first execution</SPAN></P><DIV class="c-virtual_list__item"><DIV class="c-message_kit__background c-message_kit__background--hovered p-message_pane_message__message c-message_kit__message"><DIV class="c-message_kit__hover c-message_kit__hover--hovered"><DIV class="c-message_kit__actions c-message_kit__actions--above"><DIV class="c-message_kit__gutter"><DIV class="c-message_kit__gutter__right"><DIV class="c-message_kit__blocks c-message_kit__blocks--rich_text"><DIV class="c-message__message_blocks c-message__message_blocks--rich_text"><DIV class="p-block_kit_renderer"><DIV class="p-block_kit_renderer__block_wrapper p-block_kit_renderer__block_wrapper--first"><DIV class="p-rich_text_block"><DIV class="p-rich_text_section">then subsequent executions use the last stored offset in Kafka, which is not the case here with the processor.</DIV><DIV class="p-rich_text_section">Thus only two options are&nbsp;<SPAN>"earliest" or "latest".</SPAN></DIV><DIV class="p-rich_text_section">&nbsp;</DIV><DIV class="p-rich_text_section"><SPAN>Thank you </SPAN></DIV><DIV class="p-rich_text_section">&nbsp;</DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV></DIV> Tue, 23 Apr 2024 08:46:04 GMT https://community.cloudera.com/t5/Support-Questions/How-to-consume-kafka-messages-from-a-specific-offset-in-nifi/m-p/387017#M246220 ckumar 2024-04-23T08:46:04Z