Topic XXX not present in metadata after 0000ms

[#1]

I deployed NiFi and Kafka cluster of cdp public cloud  on the azure.

And  I am trying to publish data to kafka(kafka-poc) user nifi(datahub-poc).

I am facing "Topic XXX not present in metadata after 0000ms" when I set 'Use transaction=false' 

And I am facing "Timeout expired after 0000milliseconds while awaiting  InitProducerId" after "Invaid transition attempted from state INITIALIZING to state ABORTING_TRANSACTION"  error when i set 'Use transaction=true'

Anything should I try to solve this problems?

[#2 · Accepted solution]

@Hae I am not exactly sure of your settings or the issue with transactions, but I can share what is working for me.   In my CDP Public Cloud nifi flows i use some required settings to PublishKafka.

SSL Context Service:Default NiFi SSL Context Service
Security Protocol: SASL_SSL

SASL Mechanism: PLAIN
Username: CDP Workload User Name
Password: CDP Workload User Password

Make sure you user has access to everything of course and you have set the workload password. Additionally, be patient on first processor run, sometimes takes minute or so of startup time to see SUCCESS.  Then give refresh on SMM to make sure the kafka topic is created and has the expected data.  

[#3]

It works. 

However  processor was spit "record has failed the validation on broker and hence will be rejected" with fail when if "complete,delete" set clean up policy on SMM. 

It works after change cleanup policy to "delete"

Thanks