Member since
08-20-2020
5
Posts
0
Kudos Received
0
Solutions
09-18-2020
08:26 AM
in my Nifi flows I have created 3 parameter context group each for an environment(DEV, SIT, PROD) respectively..... As of now whenever I migrate my flow to other environments , I manually select the paramater contexts based on the environment. I want to know if there is a way I can dynamically apply them without me manually selecting the parameter context groups manually Is there a way I can apply them dynamically ?
... View more
Labels:
- Labels:
-
NiFi Registry
09-10-2020
06:11 AM
@stevenmatison Just a small correction to the description, The columns in my CSV will be adding up every year. No columns will be removed from my CSV If in this year I have 100 columns next year it will be 110, so every year 10 new columns will get added . Basically I have 10 columns with the suffix of year, repeated for different years Once I validate them using a schema I will be applying transformation on these columns to make 110 columns into 20 columns according to my database. Also I got what you are saying but I want to find a dynamic way to validate the CSV . If you can help me with some mock flow which is similar to my challenge it will be of great help(Also preferably I want to achieve without using custom processors or script)
... View more
09-10-2020
05:25 AM
I have a scenario where the columns in a CSV are consistently changing, I want to validate this CSV irrespective of the changes Currently I am validating using a static schema, by placing a static schema in Schema Text property of Validate Record processor. But everytime there is a change in file format(a new column is added or removed) I have to update the schema in Schema text property of processor. Is there a way I can make this dynamic ? Currently I am using NIFI 1.11.4(I couldnt find InferSchema processor or JSON to Avro converter)
... View more
Labels:
- Labels:
-
NiFi Registry
08-24-2020
04:50 AM
Tried with alldelineatedvalues but its not working Can you share if you have any flow in which it worked for you
... View more
08-20-2020
06:39 AM
Input to my Nifi flow is from CDC-> Kafka I am using ConsumeKafka processor and in the flowfile contant I have list of columns on of a table on which CDC is enabled, Based on a flag present in my flowfile-content I am getting all the column values whihc were updated in the form of an array, I am storing this array in a attribute ex: myAttr:[123,456,789] For each flowfile this attributes is getting generated. Now, I have a static array which contains a list of allowed keys(this is not generated from flow file, its something i am using to check the values in my attribute) ex: AllowedKeys:[111,222,333,123,456,789] I am trying to see if the values inside my attribute "myAttr" contains only the allowed keys if all the values in myattr is present in AllowedKeys then i want to route it to success otherwise to failure Please let me know how can i achieve this(preferably without ExecuteScript or custom processor)
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache NiFi
-
NiFi Registry