Member since
07-16-2021
11
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2017 | 07-16-2021 05:42 AM |
07-05-2023
06:09 AM
How can I update an existing document (replacing it with the new one received) based on a specific key? the document is something similar to this one: { "field1": { "key1": "value", "key2": "value", "key3": "value" }, "key": { "subfield": { "uuid": 1234 } } } the key is key.subfield.uuid and it is contained into the json entering in the putmongo processor
... View more
Labels:
- Labels:
-
Apache NiFi
-
NiFi Registry
07-28-2021
03:09 AM
I found out this solution but when I apply it on nifi it returns an error saying " is invalid because specification not valid for the selected transformation but it works on jolttransformation demo (https://jolt-demo.appspot.com/#inception😞 [
{
"operation": "shift",
"spec": {
"*": {
"id": "id"
},
"#1": {
"&": "result"
}
}
}
] I don't understand why
... View more
07-28-2021
12:14 AM
I would like to merge two different json in nifi. first json comes from an attribute to json processor and contains just an id, something like: {
"id": "1234"
}
second one is a json of this structure: [
{"id1": "1234","id2": "1234","id3": "1234"},
{"id1": "1234","id2": "1234","id3": "1234"},
{"id1": "1234","id2": "1234","id3": "1234"},
{"id1": "1234","id2": "1234","id3": "1234"}
] desired result should be something like: {
"id": "1234",
"data": [
{"id1": "1234","id2": "1234","id3": "1234"},
{"id1": "1234","id2": "1234","id3": "1234"},
{"id1": "1234","id2": "1234","id3": "1234"},
{"id1": "1234","id2": "1234","id3": "1234"}
]
}
how can I make it? I was trying merge content to concatenate those two json flowfiles having this result: [
{
"id": "1233"
},
[
{
"id1": "1234",
"id2": "1234",
"id3": "1234"
},
{
"id1": "1234",
"id2": "1234",
"id3": "1234"
},
{
"id1": "1234",
"id2": "1234",
"id3": "1234"
},
{
"id1": "1234",
"id2": "1234",
"id3": "1234"
}
]
] now, I think I need a Jolt transform (or maybe something else ?!?) but I don't know which are the parameters of the transformation...
... View more
Labels:
- Labels:
-
Apache NiFi
07-20-2021
09:13 AM
UPDATE: I found this little trick but i need some help to understand why it does not work good. I have a json coming as response from an InvokeHTTP like this: {
"id": "1234567890",
"status": "completed",
"others": "abc",
"field": "1"
}
I would like to use a processor (maybe RouteOnAttribute??) that checks if the status of the response is completed and in case of un-match, it goes back to the invokeHttp to check status again. I tried this thing: EvaluateJsonPath(to extract id from json) -> InvokeHTTP(to get status) -> RouteOnAttribute(to check status if completed) -> match goes to a funnel -> unmatch goes to EvaluateJsonPath again (to create a loop) For the InvokeHTTP I need the original json file since I have to get the request id. I used Route to property name with the following attribute: status - ${status:equals('completed')} unfortunately, I always receive an un-match even if the status is completed. Where is the issue?
... View more
07-20-2021
03:01 AM
I have an InvokeHTTP processor that calls a long running process. The long running returns an id when called. to check if the long running process ends, I have to call another API with the returned id as argument. If the returned json contains a keyword "completed", I can call another API to get results. I would like to know how to manage this?
... View more
Labels:
- Labels:
-
Apache NiFi
07-19-2021
12:57 AM
Hi @adhishankarit, thanks for your response....one question...I don't see the link to download the xml..where i can find it?
... View more
07-16-2021
05:42 AM
it seems that the problem is related to the connection with kafka docker. In this moment I solved dockerize the NiFi installation, calling kafka as kafka:9092 from the apache nifi consumer
... View more
07-16-2021
05:41 AM
I have a json like this coming from an input port: {
"url": "blablabla",
"keyword": "foo"
} then I have to generate a new json to pass to a post call. The new json is something like this: {
"requests": [
{
"source": "blablabla",
"params": {
"keywords": [
"something"
],
"sub-field1": "1"
}
}
],
"field1": "1",
"field2": "2",
"field3": false
} where the array keywords should be replaced with a new array with the value of the previous Json ("foo"). The resulting is: {
"requests": [
{
"source": "blablabla",
"params": {
"keywords": [
"foo"
],
"sub-field1": "1"
}
}
],
"field1": "1",
"field2": "2",
"field3": false
} then i pass this json to a invokehttp to call the post. Could you please explain me how to generate this flow? thanks
... View more
Labels:
- Labels:
-
Apache NiFi
07-16-2021
02:02 AM
I'm quite new in Apache Nifi but I'm encountering some issues trying to connect a kafka microservice (with a producer) with Apache nifi consumer. Basically, I have a docker-compose like this: zookeeper:
container_name: zookeeper_test
image: wurstmeister/zookeeper #zookeeper:3.5.7
ports:
- 2181:2181
kafka:
container_name: kafka_test
image: wurstmeister/kafka #:2.13-2.6.0
ports:
- 9092:9092
environment:
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
depends_on:
- zookeeper
- kafkaui
kafkaui:
container_name: kafka-ui_test
image: provectuslabs/kafka-ui:latest
environment:
- KAFKA_CLUSTERS_0_NAME=kafka
- KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:9092
- KAFKA_CLUSTERS_0_ZOOKEEPER= zookeeper:2181
ports:
- 6789:8080
test:
container_name: test
build:
context: ./test
dockerfile: Dockerfile
depends_on:
- kafka
command: python test.py test is my producer: from kafka import KafkaProducer
import json
from time import sleep
producer = KafkaProducer(bootstrap_servers='kafka:9092')
json_message = {"hello":"world"}
for i in range(1000):
producer.send("INPUT", json.dumps(json_message).encode('utf-8'))
producer.flush()
sleep(1) Through the KafkaUI I'm able to see the topic SL.CPTI.INPUT that has been sent. In the Apache nifi dashboard, I set a ConsumerKafka_2.6 with these parameters: Kafka broker: localhost:9092 Group ID: 1 Topic: SL.CPTI.INPUT then I connected to this a funnel when "success", just to see the message received. Unfortunately, doing this, I do not see anything received. I just see a lot of tasks in the consumerkafka box but no elements in the queue connected to the funnel. I expect to see the json received, isn'it? May I miss something?
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache NiFi