question Re: Topic XXX not present in metadata after 0000ms in Support Questions

Topic XXX not present in metadata after 0000ms

Hae — Tue, 21 Apr 2026 06:45:51 GMT

I deployed NiFi and Kafka cluster of cdp public cloud on the azure.

And I am trying to publish data to kafka(kafka-poc) user nifi(datahub-poc).

I am facing "Topic XXX not present in metadata after 0000ms" when I set 'Use transaction=false'

And I am facing "Timeout expired after 0000milliseconds while awaiting InitProducerId" after "Invaid transition attempted from state INITIALIZING to state ABORTING_TRANSACTION" error when i set 'Use transaction=true'

Anything should I try to solve this problems?

Re: Topic XXX not present in metadata after 0000ms

steven-matison — Tue, 28 Nov 2023 13:55:13 GMT

@Hae I am not exactly sure of your settings or the issue with transactions, but I can share what is working for me. In my CDP Public Cloud nifi flows i use some required settings to PublishKafka.

SSL Context Service:Default NiFi SSL Context Service
Security Protocol: SASL_SSL

SASL Mechanism: PLAIN
Username: CDP Workload User Name
Password: CDP Workload User Password
 
Make sure you user has access to everything of course and you have set the workload password. Additionally, be patient on first processor run, sometimes takes minute or so of startup time to see SUCCESS.  Then give refresh on SMM to make sure the kafka topic is created and has the expected data.  

Re: Topic XXX not present in metadata after 0000ms

Hae — Wed, 29 Nov 2023 08:19:19 GMT

It works.

However processor was spit "record has failed the validation on broker and hence will be rejected" with fail when if "complete,delete" set clean up policy on SMM.

It works after change cleanup policy to "delete"

Thanks