Member since
07-10-2023
25
Posts
1
Kudos Received
0
Solutions
12-29-2023
05:25 AM
Hi @SAMSAL your solution is workink, thank you so much!
... View more
12-18-2023
06:06 AM
Is the problem that the other system doesnt take negative values?
... View more
12-18-2023
05:01 AM
@SAMSAL Thank you for help, everything works unfortunately except the date, but as you said, it is more complicated
... View more
10-23-2023
11:44 AM
@MWM @cotopaul If you get the record reader/writer using the schema(s) you want, you do not have to do any magic to convert values, it should just work. Only use, inferSchema long enough to get the structure when you have none. Then copy/paste it and use it as @cotopaul has described in place of InferSchema. You can also use Schema Registry. Make the edits you need to satisfy reader (upstream), writer (downstream) as they are sometimes needing minor adjustments like in this case.
... View more
08-04-2023
01:37 PM
@MWM MergeContent is probably not the processor you want to use to merge your JSON files as it will simply concatenate them together instead of creating one larger json. You should use the MergeRecord processor instead. In this processor you can specify a JsonTreeReader to read your source Json files and then use a CSVRecordSetWriter to output merged CSV file(s). The rest of the MergeRecord processor configuration will control how many source Json records are added in to a single output CSV. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
08-01-2023
06:54 AM
1 Kudo
Hi @MWM , What you are describing is a classical data enrichment pattern that can be achieved using ForkEnrichment & JoinEnrichment processors. For more information on this please refer to : https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.20.0/org.apache.nifi.processors.standard.JoinEnrichment/additionalDetails.html Based on your scenario the "SQL" strategy of the JoinEnrichment will work the best for you since you can select the main fields (system, name, surname, phone, mail) from the original flowfile data and select proffesion and department from enrichment result: SELECT o.system, o.name, e.surname,o.phone, o.mail, e.profession, e.department FROM original o LEFT OUTER JOIN enrichment e ON o.name= e.name Since you are splitting the CSV and enrich per record then you can just join by name. If you have an API where you can get a collection of user information then you dont have to split and you can do the enrichment on multiple records from the CSV vs. returned records from the API json output , however be aware that if you have large data set this strategy "... can lead to heap exhaustion and cause stability problems or OutOfMemoryErrors to occur". Please review the link above to see how this can be mitigated. If you find this is helpful please accept solution. Thanks
... View more
07-12-2023
06:30 AM
@MWM Your description is not super clear to me. FlowFile Attributes are in format of <attribute name>=<attribute value> example key=value FlowFile attributes: Attribute name = Attribute value
name=toyota
type=car Here is how I interpret what you have provided: - You have a FlowFile that has a custom FlowFile "Attribute Name" on it "name". - That "name" Attribute has no value associated to it. (name = ) The UpdateAttribute processor allows you to add new key=value (attribute name = attribute value) attribute pairs to an existing FlowFile or modify the value of an existing FlowFile Attribute Name. You can NOT create a new attribute and then modify or use that new attribute within the same processor in which it was created. Lets say you have a FlowFile with the example FlowFile attributes and values I shared above. Then you use the UpdateAttribute processor to change the value on the Attribute "name" to all uppercase. The NiFi Expression Language (NEL) statement you created: ${name:toUpper()} will take the subject "name" and return it's value "toyota". That returned value is passed to the toUpper() NEL function which converts the it to Uppercase "TOYOTA". The resulting value is then written to the FlowFile attribute defined in the UpdateAttribute processor. Assuming an UpdateAttribute processor configured as below: The NEL expression resulting a new value of "TOYOTA" would be written to the FlowFile Attribute "name" since "name" already exists as an attribute on the FlowFile, its current attribute value of "toyota" would be replaced with new value "TOYOTA". Hope this helps clarify how NiFi FlowFile Attributes and the UpdateAttribute processor is used. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
07-11-2023
06:22 AM
@MWM For good security reasons, NiFi does not support passwordless keystore and truststore files. The password fields can not be blank. Also keep in mind that the default "nifi.web.https.host=127.0.0.1" is localhost on the machine were you installed NiFi. So you will not be able to access this IP from any other machine. You'll need to use a browser installed on the same machine. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more