Apache nifi and kafka microservice

[#1]

I'm quite new in Apache Nifi but I'm encountering some issues trying to connect a kafka microservice (with a producer) with Apache nifi consumer.

Basically, I have a docker-compose like this:

zookeeper:
  container_name: zookeeper_test
  image: wurstmeister/zookeeper #zookeeper:3.5.7
  ports:
  - 2181:2181

kafka:
  container_name: kafka_test
  image: wurstmeister/kafka #:2.13-2.6.0
  ports:
  - 9092:9092
  environment:
      KAFKA_ADVERTISED_HOST_NAME: kafka
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
  depends_on: 
      - zookeeper
      - kafkaui

kafkaui:
  container_name: kafka-ui_test
  image: provectuslabs/kafka-ui:latest
  environment: 
      - KAFKA_CLUSTERS_0_NAME=kafka
      - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:9092
      - KAFKA_CLUSTERS_0_ZOOKEEPER= zookeeper:2181
  ports:
      - 6789:8080

test:
  container_name: test
  build:
      context: ./test
      dockerfile: Dockerfile
  depends_on:
      - kafka
  command: python test.py

test is my producer:

from kafka import KafkaProducer
import json
from time import sleep

producer = KafkaProducer(bootstrap_servers='kafka:9092')
json_message = {"hello":"world"}


for i in range(1000):
   producer.send("INPUT", json.dumps(json_message).encode('utf-8')) 
   producer.flush()
   sleep(1)

Through the KafkaUI I'm able to see the topic SL.CPTI.INPUT that has been sent.

In the Apache nifi dashboard, I set a ConsumerKafka_2.6 with these parameters: Kafka broker: localhost:9092 Group ID: 1 Topic: SL.CPTI.INPUT

then I connected to this a funnel when "success", just to see the message received. Unfortunately, doing this, I do not see anything received. I just see a lot of tasks in the consumerkafka box but no elements in the queue connected to the funnel. I expect to see the json received, isn'it? May I miss something?

[#2 · Accepted solution]

it seems that the problem is related to the connection with kafka docker. In this moment I solved dockerize the NiFi installation, calling kafka as kafka:9092 from the apache nifi consumer

[#3]

@alexmarco it seems like you resolved your issue. If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.