Member since
05-17-2021
4
Posts
0
Kudos Received
0
Solutions
06-07-2021
08:34 AM
@Acbx It looks like your CSV uses commas as the field delimiter. So the solution i provided parses the entire file line by line and changes all "." to ",". So, I am guessing that you have other places within your CSV that also had ".", thus creating the additional 5 field columns. Are trying to create a new column for cents? Is that why you are changing 109.29 tp 109,29? If you are not looking for a new column, how will downstream system parse this edited CSV now that you added a new comma in there? You could write a complex Java regular expression in the Search Value to match only specifically on column number X (Money Column) and then use Replacement Strategy "Regex Replace" to edit it. Let's assume the "Money" Column was column number 5. And then wrap money once converted from 109.29 to 109,29 in quotes so it is not treated as two columns later on.... Search Value: ^(.*?),(.*?),(.*?),(.*?),(.*?),(.*?)$ Replacement Value: $1,$2,$3,$4,"${'$5':replace(".",",")}",$6 So above would manipulate column 5 only and change 109.29 in to "109,29". Hope this helps you, Matt
... View more
05-21-2021
03:13 PM
1 Kudo
Hi, Try using the replace text processor in "Literal Replace" mode for the Replacement Strategy setting. It's no obvious, but I have placed a single space in the Search Value setting. Alternatively, you could explore using an Avro schema to achieve this, and this links in nicely with your other question too. For example, given a flow file with the content (I'm using the GenerateFlowFile to create this): Using this flow as a test: I've created content in the GenerateFlowFile of: with an attribute called avro.schema: Note the highlighted "name" attribute which defines the name you need on the output, and the "aliases" array value which is the input name value. For reference, here's the GenerateFlowFile configuration: The ConvertRecord has the following CSVReader configured: And using an otherwise normally configured CSVRecordSetWriter I get the following output: Also note that I have also changed the case using this approach, in fact you can rename the column to whatever you need. I found working with Avro schemas a bit of a steep learning curve, but definitely worth the time investment. I mention this in particular as it compliments your other question about date formats very well. Hope that helps
... View more
05-21-2021
02:23 PM
Hi, I've just noticed your date strings in the sample aren't consistent which will make things difficult I think. For example in line 1 your OrderDate format is "M/dd/yyyy" (single digit month) and in ship date it seems to be "MM/dd/yyyy" (double digit month). For the CSV to work correctly all of your date fields would need to adhere to the same format I believe. You can do a couple of things to resolve this: Fix it in the source. The best option in my opinion, if possible. Clean the data in NiFi before it arrives at the PutDatabaseRecord processor. Unfortunately the second is a touch beyond my current level of expertise, but if I was you I would explore either ReplaceText processor with the appropriately regex expression, or a an UpdateRecord processor with a SQL like statement to update this field in the flowfile - note though this would would require a separate CSVReader service that reads your data as strings. Good luck.
... View more