Member since
07-16-2021
11
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 3164 | 07-16-2021 05:42 AM |
07-19-2021
12:57 AM
Hi @adhishankarit, thanks for your response....one question...I don't see the link to download the xml..where i can find it?
... View more
07-16-2021
05:42 AM
it seems that the problem is related to the connection with kafka docker. In this moment I solved dockerize the NiFi installation, calling kafka as kafka:9092 from the apache nifi consumer
... View more
07-16-2021
05:41 AM
I have a json like this coming from an input port: {
"url": "blablabla",
"keyword": "foo"
} then I have to generate a new json to pass to a post call. The new json is something like this: {
"requests": [
{
"source": "blablabla",
"params": {
"keywords": [
"something"
],
"sub-field1": "1"
}
}
],
"field1": "1",
"field2": "2",
"field3": false
} where the array keywords should be replaced with a new array with the value of the previous Json ("foo"). The resulting is: {
"requests": [
{
"source": "blablabla",
"params": {
"keywords": [
"foo"
],
"sub-field1": "1"
}
}
],
"field1": "1",
"field2": "2",
"field3": false
} then i pass this json to a invokehttp to call the post. Could you please explain me how to generate this flow? thanks
... View more
Labels:
- Labels:
-
Apache NiFi
07-16-2021
02:02 AM
I'm quite new in Apache Nifi but I'm encountering some issues trying to connect a kafka microservice (with a producer) with Apache nifi consumer. Basically, I have a docker-compose like this: zookeeper:
container_name: zookeeper_test
image: wurstmeister/zookeeper #zookeeper:3.5.7
ports:
- 2181:2181
kafka:
container_name: kafka_test
image: wurstmeister/kafka #:2.13-2.6.0
ports:
- 9092:9092
environment:
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
depends_on:
- zookeeper
- kafkaui
kafkaui:
container_name: kafka-ui_test
image: provectuslabs/kafka-ui:latest
environment:
- KAFKA_CLUSTERS_0_NAME=kafka
- KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:9092
- KAFKA_CLUSTERS_0_ZOOKEEPER= zookeeper:2181
ports:
- 6789:8080
test:
container_name: test
build:
context: ./test
dockerfile: Dockerfile
depends_on:
- kafka
command: python test.py test is my producer: from kafka import KafkaProducer
import json
from time import sleep
producer = KafkaProducer(bootstrap_servers='kafka:9092')
json_message = {"hello":"world"}
for i in range(1000):
producer.send("INPUT", json.dumps(json_message).encode('utf-8'))
producer.flush()
sleep(1) Through the KafkaUI I'm able to see the topic SL.CPTI.INPUT that has been sent. In the Apache nifi dashboard, I set a ConsumerKafka_2.6 with these parameters: Kafka broker: localhost:9092 Group ID: 1 Topic: SL.CPTI.INPUT then I connected to this a funnel when "success", just to see the message received. Unfortunately, doing this, I do not see anything received. I just see a lot of tasks in the consumerkafka box but no elements in the queue connected to the funnel. I expect to see the json received, isn'it? May I miss something?
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache NiFi