Hi Team,
We have configured HDF 3.1.1 single node instance for POC . He we are trying to perform a small test :
We have written a small python program to generate kafka message :
from kafka import KafkaProducer
from kafka.errors import KafkaError
producer = KafkaProducer(bootstrap_servers=' hostname:6667')
topic = "jsontest"
producer.send(topic, b'test message')
We want to consume this message thorough Nifi Consumekafka ,which is not working .
nifi.app.log
(type=HEARTBEAT, length=2086 bytes) from hdfdemo.c.corecompetetraining.internal:9090 in 1 millis
2018-05-15 07:51:19,808 INFO [Clustering Tasks Thread-2] o.a.n.c.c.ClusterProtocolHeartbeater Heartbeat created at 2018-05-15 07:51:19,804 and sent to hdfdemo.c.corecompetetraining.internal:9088 at 2018-05-15 07:51:19,808; send took 4 millis
2018-05-15 07:51:20,352 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available.
2018-05-15 07:51:21,505 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available.
2018-05-15 07:51:22,357 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available.
2018-05-15 07:51:23,560 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available.
2018-05-15 07:51:24,557 INFO [Heartbeat Monitor Thread-1] o.a.n.c.c.h.AbstractHeartbeatMonitor Finished processing 1 heartbeats in 8392 nanos
2018-05-15 07:51:24,668 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available.
2018-05-15 07:51:24,811 INFO [Process Cluster Protocol Request-8] o.a.n.c.p.impl.SocketProtocolListener Finished processing request 8aec83a3-cfbc-43c2-918d-143d701d13a8 (type=HEARTBEAT, length=2086 bytes) from hdfdemo.c.corecompetetraining.internal:9090 in 1 millis
2018-05-15 07:51:24,815 INFO [Clustering Tasks Thread-2] o.a.n.c.c.ClusterProtocolHeartbeater Heartbeat created at 2018-05-15 07:51:24,809 and sent to hdfdemo.c.corecompetetraining.internal:9088 at 2018-05-15 07:51:24,815; send took 6 millis
2018-05-15 07:51:25,582 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available.
2018-05-15 07:51:26,735 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available.
2018-05-15 07:51:27,888 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available.
2018-05-15 07:51:28,941 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available.
2018-05-15 07:51:29,558 INFO [Heartbeat Monitor Thread-1] o.a.n.c.c.h.AbstractHeartbeatMonitor Finished processing 1 heartbeats in 7009 nanos
2018-05-15 07:51:29,817 INFO [Process Cluster Protocol Request-9] o.a.n.c.p.impl.SocketProtocolListener Finished processing request fcae4257-6a34-4691-9427-530ed061faf1 (type=HEARTBEAT, length=2086 bytes) from hdfdemo.c.corecompetetraining.internal:9090 in 1 millis
2018-05-15 07:51:29,819 INFO [Clustering Tasks Thread-2] o.a.n.c.c.ClusterProtocolHeartbeater Heartbeat created at 2018-05-15 07:51:29,815 and sent to hdfdemo.c.corecompetetraining.internal:9088 at 2018-05-15 07:51:29,819; send took 3 millis
2018-05-15 07:51:29,944 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available.
2018-05-15 07:51:30,846 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available.
2018-05-15 07:51:31,999 WARN [Timer-Driven Process Thread-5] org.apache.kafka.clients.NetworkClient [Consumer clientId=consumer-11, groupId=test] Connection to node -1 could not be established. Broker may not be available
nifi-user.log
2018-05-15 08:06:11,913 INFO [NiFi Web Server-387] org.apache.nifi.web.filter.RequestLogger Attempting request for (anonymous) GET http://hostname:9090/nifi-api/flow/status (source ip: )
Please help on this .
Regards,
Vishal
