Member since
01-05-2017
153
Posts
10
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2665 | 02-20-2018 07:40 PM | |
1640 | 05-04-2017 06:46 PM |
06-16-2017
02:37 PM
Can you provide a link about "Flow status querying"? I can't seem to find anything on it.
... View more
06-15-2017
02:08 PM
1 Kudo
So, I have a situation where I am using MiniFi and as we know there is no user interface for MiniFi. I think the queues are backed up in the flows on there but theres no way for me to tell and if it is, I cannot figure out how to clear them? Can anyone give me any help?
... View more
Labels:
- Labels:
-
Apache MiNiFi
06-13-2017
02:52 PM
We figured it out Bryan. We didn't have a message demarcator set and once we set it, the error went away! Thank you!
... View more
06-13-2017
02:34 PM
Thanks Bryan I think you have led me to where I need to go. I think the configurable property is message.max.bytes that needs to be changed in the config for the broker. I thought I could do that in Ambari but I can't find that property there so I am trying to hunt where to change that property.
... View more
06-13-2017
12:42 PM
Thanks Bryan. Just for my understanding, these aren't events right? They are messages? What is a message, in terms of this kind of error? I was thinking this was a single log file event that was over 2 MB or whatever. But now I suspect its different. Can you help me understand?
... View more
06-12-2017
05:54 PM
Im starting to suspect its something to do with either MergeContent or the SplitText we're using.
I've noticed that when I switch SplitText to Line Count 1 it doesn't happen but then we get terrible bottlenecks that break the system. We currently have it at 15 but this produces the error in the PublishKafka_0_10. Also when I checked the actual data provenance for Publish_Kakfa_0_10, I saw that it was producing files of size 2.5+ MB which makes me think its MergeContent. Here are the screenshots
... View more
06-12-2017
05:39 PM
Yeah I just tried to change it to 2 MB and got the error shown in my response above.
... View more
06-12-2017
05:38 PM
I updated the Max Request Size in Publish_Kafka_0_10 to 2 MB and it produced a different error related to the it being a message larger than the server will accept but doesn't list the size of the limitation anymore. This one is a RecordTooLargeException while the last one was a TokenTooLargeException.
... View more
06-12-2017
03:01 PM
From what I understand according to: https://community.hortonworks.com/questions/70355/nifi-putkafka-processor-maximum-allowed-data-size.html Publish_Kakfa_0_10 was configured to receive messages larger than 1 MB but I am still getting the same error on this (see screenshot) Does anyone know if they fixed this in the Publish_Kafka_0_10 processor? My Nifi version is 1.1.2
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache NiFi
06-12-2017
02:55 PM
Not sure what your second comment meant if it was for me or not. I'm dealing with Nifi throughout the whole stack from the raw log files on a remote server through the Remote Process Group into the Input Port and into the HDFS. I guess that means I'm not a "user".
... View more