Member since
05-16-2021
8
Posts
2
Kudos Received
0
Solutions
06-21-2021
11:35 AM
I'm having a similar issue. The SFTP processors are also not being able to connect. (also on jre 😎 One thing that might help, is that running nifi using the official docker image, works fine. And, the JRE on that container is also the jre8 but not the same exact version, the version on the container is a bit older than mine. Haven't had the time to test this out, but it might help; Also, on local docker and on local host, "it works on my machine". The faulty one is on AWS Have you guys achieved any progress on the issue?
... View more
06-03-2021
06:32 AM
Hi @MattWho , Thanks for the response. I'll happily create a feature request, though I wasn't sure if I was missing something obvious that would meet my objective/requirements. Thank you for clarifying, I'll go take a look at the links you provided (admittedly I had missed the mailing lists, oops). Kind regards Mark
... View more
05-21-2021
03:26 PM
1 Kudo
There isn't enough information in your question to provide you with an accurate answer. However, note that the FetchFile processor is intended to work with an upstream ListFile processor, which provides you with the opportunity to specify a location and other filtering criteria such as file name patterns or particular sub folders (path pattern) amongst other options. By default, the ListFile processor outputs one empty FlowFile with attributes related to each file it identifies, these are then used as an input for the FetchFile so it knows which files to retrieve. Also, the ListFile is aware of the of its last listing, and will only list files younger than this on it's next execution. Hope that helps. The List & Fetch pattern is very common, however should you have simpler requirements, the GetFile processor may be suitable.
... View more
05-21-2021
03:13 PM
1 Kudo
Hi, Try using the replace text processor in "Literal Replace" mode for the Replacement Strategy setting. It's no obvious, but I have placed a single space in the Search Value setting. Alternatively, you could explore using an Avro schema to achieve this, and this links in nicely with your other question too. For example, given a flow file with the content (I'm using the GenerateFlowFile to create this): Using this flow as a test: I've created content in the GenerateFlowFile of: with an attribute called avro.schema: Note the highlighted "name" attribute which defines the name you need on the output, and the "aliases" array value which is the input name value. For reference, here's the GenerateFlowFile configuration: The ConvertRecord has the following CSVReader configured: And using an otherwise normally configured CSVRecordSetWriter I get the following output: Also note that I have also changed the case using this approach, in fact you can rename the column to whatever you need. I found working with Avro schemas a bit of a steep learning curve, but definitely worth the time investment. I mention this in particular as it compliments your other question about date formats very well. Hope that helps
... View more
05-21-2021
02:23 PM
Hi, I've just noticed your date strings in the sample aren't consistent which will make things difficult I think. For example in line 1 your OrderDate format is "M/dd/yyyy" (single digit month) and in ship date it seems to be "MM/dd/yyyy" (double digit month). For the CSV to work correctly all of your date fields would need to adhere to the same format I believe. You can do a couple of things to resolve this: Fix it in the source. The best option in my opinion, if possible. Clean the data in NiFi before it arrives at the PutDatabaseRecord processor. Unfortunately the second is a touch beyond my current level of expertise, but if I was you I would explore either ReplaceText processor with the appropriately regex expression, or a an UpdateRecord processor with a SQL like statement to update this field in the flowfile - note though this would would require a separate CSVReader service that reads your data as strings. Good luck.
... View more