Member since
04-11-2016
469
Posts
320
Kudos Received
118
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1696 | 03-09-2018 05:31 PM | |
2040 | 03-07-2018 09:45 AM | |
2030 | 03-07-2018 09:31 AM | |
3537 | 03-03-2018 01:37 PM | |
2002 | 10-17-2017 02:15 PM |
02-09-2021
10:41 AM
Here is a cool NiFi websocket app https://www.datainmotion.dev/2020/12/ingesting-websocket-data-for-live-stock.html Hosting web apps in NiFi https://www.datainmotion.dev/2020/11/flank-smart-weather-websocket.html https://www.datainmotion.dev/2020/12/ingesting-websocket-data-for-live-stock.html
... View more
01-28-2021
03:26 AM
1 Kudo
Hi @pvillard, I have some doubts about what kind of keytab file to create. I don't know much about Kerberos and do not find much information about the kind of Keytab file I should use to connect my Nifi Cluster to a kerberized Cloudera Cluster. Can you help me?
... View more
09-21-2020
05:53 AM
1 Kudo
@dvmishra It always best to start a new thread/question rather than adding a new question to an existing thread that already has an excepted answer. As far as being able to reset the offset for a specific consumer group from within NiFi itself, this is not something that can be done via the ConsumeKafka processors. The offset is not stored by NiFi. Offsets for each consumer group are stored in Kafka. Would not make make much sense to build such an option in to a NiFi processor if it was possible. Every time the processor executes it would reset in that case which is probably not the desired outcome. There are numerous threads online around reseting the offset in Kafka you may want to explore. Here are a couple: https://gist.github.com/marwei/cd40657c481f94ebe273ecc16601674b https://gist.github.com/mduhan/0e0a4b08694f50d8a646d2adf02542fc If you can figure out how to accomplish reset via a custom script of external command, NiFi does offer several script execution and command line execution processors. You may be able to use these processors to execute your script to rest the offset in Kafka. Aside from above, you can change the "group id" (new consumer group) and change the "offset reset" to "earliest". Then restart processor to start consuming topic form beginning again as a different consumer group. Hope this helps, Matt
... View more
03-31-2020
07:16 AM
@pvillard How does this work exactly? Im having issues segmenting large files as well. When i split them do i do it multiple times or just once and then I can recombine them successively. Thanks for you help!
... View more
02-17-2020
03:18 AM
Hi, can you explain how did you solve this problem?
... View more
01-31-2020
11:10 AM
Try using SELECTHIVEQL processor for your hive connection and Hive query instead of ExecuteSQL or QueryDatabase
... View more
01-28-2020
08:14 PM
Hi @pvillard, Would you happen to have a sample template for this process? Thanks Tim
... View more
12-23-2019
09:55 PM
@MattWho , Thank you for the details. I can understand your point that same file cannot be accessed across cluster nodes. But, I am using Nifi is single node (without cluster) and was thinking that this should work. Yes, i do use "Update Attribute" with the below file name conventions. This generates separate flow file for every message. I am trying to have this as one single file per node. ${filename:replace(${filename},"FileName_"):append(${now():format("yyyy-MM-dd-HH-mm-ss")})}.Json Thank you
... View more
12-04-2019
03:48 AM
What if I need to send a binary file? I have a zip file that needs to be sent
... View more
11-14-2019
10:59 PM
Hi, I can see significant impact on the hive query after implementation of Ranger and using mysql 5.7. How can we reduce the response time added due to the Ranger.
... View more