Member since
02-01-2022
274
Posts
97
Kudos Received
60
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
401 | 05-15-2025 05:45 AM | |
3396 | 06-12-2024 06:43 AM | |
5925 | 04-12-2024 06:05 AM | |
4055 | 12-07-2023 04:50 AM | |
2178 | 12-05-2023 06:22 AM |
12-01-2023
06:05 AM
@I_Mesh01 This error is stating that your topic schema and flink schema do not match. I cant see much above related to the shape of the data or the schema, but this is what you should be looking at. Conflict in data/column types, missing data, etc, will result in the deserialization errors.
... View more
12-01-2023
05:54 AM
@matdulgi The drivers are able to be downloaded from public internet and you are able to use them accordingly. https://www.cloudera.com/downloads/connectors/impala/odbc/2-7-0.html https://www.cloudera.com/downloads/connectors/impala/jdbc/2-6-32.html.html
... View more
11-29-2023
04:39 AM
CHeck out this link @joseomjr shared in another post: https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version2.0.0-M1 And here is a great blog from Pierre Villard: https://medium.com/cloudera-inc/getting-ready-for-apache-nifi-2-0-5a5e6a67f450 Nifi 2.0 brings python native, where jython was very limited library set.
... View more
11-29-2023
12:19 AM
It works. However processor was spit "record has failed the validation on broker and hence will be rejected" with fail when if "complete,delete" set clean up policy on SMM. It works after change cleanup policy to "delete" Thanks
... View more
11-28-2023
05:47 AM
@joseomjr Is on to the right solution here. Your regex statement should match "kafka.topic" not "${kafka.topic}". A quick test in regex101.com confirms "kafka\.topic" should match.
... View more
11-15-2023
05:36 AM
1 Kudo
In regards to your nifi expression language.: Test in a simple flow, and inspect the flowfile's attributes to insure topicName is correct. Then take it to the kafka processor. If you are not seeing the results of the expression, something isnt right, or the value doesnt except Expression Language. Look at the (?) on any property to see what is accepted. I suspect the expression language is invalid as well, so make sure you test and confirm the attribute is as expected before trying to use it deeper in your flow. In regards to mapping. What i suggest could be DistributedMapCache or even a flow that does a lookup against some other service. With this concept you provide a mapping key value pairs that correspond to your consumer and producer topics. When you lookup a key, For Example: "alerts_Consumer_Topic_Name" the value would be "alerts_Producer_Topic_Name". If this is stored outside of nifi, then these values can be managed and changed outside of the scope of the nifi flow. Example flow with DistributeMapCache: https://github.com/ds-steven-matison/NiFi-Templates/blob/main/DistributedCache_Demo.xml
... View more
11-13-2023
10:39 PM
We did this at our end and ended up re-cycling the provenance repository much faster than usual. The huge amount of data that an output of a tailfile generates can fill up both your content and provenance repositories.
... View more
11-13-2023
07:15 AM
@Arash Did you inspect the flowfile (assure its the expected format, etc) and inspect the flowfile attributes (may be more detail into the conflict). Additionally, you can set the processor log level to DEBUG and/or monitor the nifi-app.log for more details. If this worked before, and now doesnt, i would expect something to have changed in the flowfile.
... View more
11-08-2023
07:42 AM
@Arash This error is indicating an issue with your SSL Context Service "SSL Service for elasticsearch cluster". If this was previously working, perhaps the SSL cert has changed? You should check if the elasticsearch cert has been renewed, and if so, update the keystores/truststores accordingly and test again.
... View more
10-23-2023
11:44 AM
@MWM @cotopaul If you get the record reader/writer using the schema(s) you want, you do not have to do any magic to convert values, it should just work. Only use, inferSchema long enough to get the structure when you have none. Then copy/paste it and use it as @cotopaul has described in place of InferSchema. You can also use Schema Registry. Make the edits you need to satisfy reader (upstream), writer (downstream) as they are sometimes needing minor adjustments like in this case.
... View more