Member since
04-20-2016
27
Posts
5
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
6909 | 09-30-2016 03:06 PM |
09-30-2016
09:42 PM
@Bryan Bende I appreciate the help immensely will get back to you on Monday with feedback. Cheers to a good weekend!
... View more
09-30-2016
03:06 PM
@Bryan Bende your take on my comments above is much appreciated. Thanks
... View more
09-29-2016
09:13 PM
@Bryan Bende let's say I want to go the route where I use different ExtractText to handle different delimiters how I do about that?Am quite confused here(a vivid example will ve helpful). From my understanding, ExtractText processor will parse a file regardless of the delimiters of the file but what actually matters is the regular expression used to extract the data?correct me if wrong. I also tried replicating your example above, the ingestion of the flow file was successful but there was no data in data the databases tables.
... View more
09-29-2016
03:27 PM
@Bryan Bende your input is much appreciated in regards to this question. Thanks
... View more
09-29-2016
12:34 AM
@Bryan Bende Thanks once again for getting back.I prefer answering your questions in this pattern: (1) Since your flow was working you must have already configured ExtractText with a pattern to parse the line right? Yes I used regular expression(attached config image below)to parse the line but I do not think that is the best way to handle this.I would rather like to parse the lines using the delimiter. (2)So are you just asking how to handle more delimiters? Yes that will helpful.
... View more
09-28-2016
03:46 PM
Hi @Bryan Bende thanks for getting back at me.To answer your question: What do you want to do with your text files? I want to be able to move the flat files from sftp containing delimiters into a sql database using nifi. How nifi handles the delimiters in the flat file are of concern to me? Which one of the processors stated above is handling the delimiters in the text file? I posted a sample data flow of what I want to achieve and it been answered by you here: https://community.hortonworks.com/questions/57779/how-to-preventing-duplicates-when-ingesting-into-m.html#comment-57785 Do you want to convert it to another format, if so what format? No I do not want to convert it into another format.
... View more
09-27-2016
10:28 PM
I have text files that sometimes have various delimiters such as quotation marks, commas, and tabs etc what processor can I use to handle such delimiters and how to I configure it to handle such delimiters in my file? ConvertCsvToAvro "properties" section has similar properties I want to achieve.Thanks
... View more
Labels:
- Labels:
-
Apache NiFi
09-21-2016
09:25 PM
@Bryan Bende thanks a lot for the help works perfect. I attached a snippet of the updated workflow should anyone experience such an issue in the future.Thanks again.
... View more
09-21-2016
09:01 PM
1 Kudo
I have a dataflow that ingest file from sftp into mysql and would like to know how to prevent an enormous amount of duplicates being ingested by nifi into mysql. Attached details below. Thanks (1)Data flow (2) count after nifi ingest into mysql (3)Original Data on SFTP
... View more
Labels:
- Labels:
-
Apache NiFi
09-21-2016
01:35 AM
@Pierre Villard sorry for the late reply. How do I verify am not ingesting multiple times the same file with List/FetchSFTP?
... View more