@Bryan Bende I appreciate the help immensely will get back to you on Monday with feedback. Cheers to a good weekend! ... View more

@Bryan Bende let's say I want to go the route where I use different ExtractText to handle different delimiters how I do about that?Am quite confused here(a vivid example will ve helpful). From my understanding, ExtractText processor will parse a file regardless of the delimiters of the file but what actually matters is the regular expression used to extract the data?correct me if wrong. I also tried replicating your example above, the ingestion of the flow file was successful but there was no data in data the databases tables. ... View more

@Bryan Bende your input is much appreciated in regards to this question. Thanks ... View more

@Bryan Bende Thanks once again for getting back.I prefer answering your questions in this pattern: (1) Since your flow was working you must have already configured ExtractText with a pattern to parse the line right? Yes I used regular expression(attached config image below)to parse the line but I do not think that is the best way to handle this.I would rather like to parse the lines using the delimiter. (2)So are you just asking how to handle more delimiters? Yes that will helpful. ... View more

Hi @Bryan Bende thanks for getting back at me.To answer your question: What do you want to do with your text files? I want to be able to move the flat files from sftp containing delimiters into a sql database using nifi. How nifi handles the delimiters in the flat file are of concern to me? Which one of the processors stated above is handling the delimiters in the text file? I posted a sample data flow of what I want to achieve and it been answered by you here: https://community.hortonworks.com/questions/57779/how-to-preventing-duplicates-when-ingesting-into-m.html#comment-57785 Do you want to convert it to another format, if so what format? No I do not want to convert it into another format. ... View more

@Bryan Bende thanks a lot for the help works perfect. I attached a snippet of the updated workflow should anyone experience such an issue in the future.Thanks again. ... View more

@Pierre Villard sorry for the late reply. How do I verify am not ingesting multiple times the same file with List/FetchSFTP? ... View more