1973
Posts
1225
Kudos Received
124
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1769 | 04-03-2024 06:39 AM | |
2756 | 01-12-2024 08:19 AM | |
1529 | 12-07-2023 01:49 PM | |
2283 | 08-02-2023 07:30 AM | |
3121 | 03-29-2023 01:22 PM |
10-29-2020
04:02 AM
Excellent tutorial !! I've downloaded your ImageProcessor processor and it worked just fine. I see you are using an ExtractImageMetadata processor in the end of the download image flow. Is it another custom processor you have built? If so, can you share the github repo, please? Thank you so much, best regards from Brazil!
... View more
10-22-2020
11:18 AM
Thanks, Tim, my whole Idea is, developers should be able to replay the message from the provenance for at least 5 days as per the requirements I'm assuming the only solution is BUMP up the provenance storage to achieve replay capability. please let me know your thoughts!.
... View more
10-12-2020
10:43 PM
Hi @Kumar78, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question.
... View more
10-09-2020
07:16 AM
1 Kudo
https://www.datainmotion.dev/2020/09/devops-working-with-parameter-contexts.html download the flow/backup up/store in git copy a flow to archive remove from production https://www.datainmotion.dev/2019/11/nifi-toolkit-cli-for-nifi-110.html
... View more
10-07-2020
02:18 PM
Thanks a lot @TimothySpann for your time and insight. Your advice saves me from future futile efforts to hack/mess up with guava jar libs. I will probably try different approach to load Kafka topics into Hive. Thanks again.
... View more
09-29-2020
02:10 PM
Hi @vineet_harkut , To check if the required parameters are valid in MongoDB processor, what I would do is Suppose, there is a query like {"key":"someValue","name":"someOtherValue"}. And "someValue" and "someOtherValue" are values inside the attributes of the incoming flowFile with attribute-names "key" and "name" respectively. Hence, the resulting query in the GetMongo processor will be {"key":"${key}","name":"${name}"}. Before using the GetMongo processor, I would use RouteOnAttribute processor to check if the input to the query is a valid one. If the input is correct, I will make a query to the GetMongo processor. If the document for the query is not found, the GetMongo processor will return empty document (empty string) as content in the flowfile only when Send Empty Result is set to true inside the GetMongo processor. Later, when I use EvaluateJsonPath processor, the flowfile will be routed to "failure" when the flowfile is empty, as empty string is not a valid JSON or to "matched" if the match for the given jsonPath is found. This will put the same content inside the flowfile as $ corresponds to entire JSON object that we are getting from the input flowfIle. Now, you can add the remaining logic.
... View more
09-15-2020
08:22 PM
@TimothySpann thanks for your response. - i assume ConsumeKafkaRecord_2_0 1.9 will consume messages from the Kafka partitions based on "max poll records".in my case I have set it to 100000. please let me know if my assumption is wrong. could you also confirm if there are any limitations to pull bulk messages like at least 10K messages in a single poll? with 1.9 nifi message size is 2 MB, Avro format and 3 topics have 2 million messages each
... View more
09-09-2020
09:13 AM
Hi @Debangshu It worked with 1.10.0 and 1.11.3, thanks mate for the resolution. Thanks David
... View more
08-19-2020
10:01 AM
https://www.datainmotion.dev/2020/08/deleting-schemas-from-cloudera-schema.html
... View more
08-06-2020
06:11 AM
That's good news. That's a tricky one. Glad things are working for you.
... View more