Hello everyone,   i have data which has only one value per line, often not even a header. My goal is to conert the data to csv and format it afterwards (remove the comments and add some columns).   Example base data: #This is my base data# ################# value1 value2 value3   Example finished data: value-description, source, timestamp value1, source1, 984359345 value2, source1, 98249732 value3, source1, 984u9834   Right now I'm able to achieve the output if I have an input which is already in a CSV format (comma separated). But if I try is with the example data above the output is the following:   header-descriptionvalue1value2value3   This happens after the ReplaceText processor which should delete blank lines. The source and timestamp column is not yet added. This is the config of the processor:   I am glad about every response.    Best regards Maurice     ... View more

Hello community,     I started using Apache NIFI for my bachelor-thesis. The basics of the data flow are already working. But there are some cases I can not really get the grasp on.    I get my files via HTTP, and they are mostly in TXT, CSV or XML.   How my workflow (data flow?) should look like: - Multiple data sources (Question 1) - Splitting the values in multiple lines - Adding a timestamp as a column to each line (Question 2) - Adding the source (name) as a column to each line (Question 2) - Checking if the value was already seen (Question 3) - Adding a new column to each line with the value "already seen" or "first seen" (Question 3) - Merging the content - Changing Filename - PutFile (Question 4)   Question 1 Do I need to make a new Data flow for each new resource? Because otherwise they have all the same or a totally random file name at the end.   Question 2 If I add a column with the same value to each line, is it better to add the value before, at or after splitting the text?   Question 3 Right now my data gets saved in separate files, for example: dat_feed1.csv, data_feed2.csv. How do I check if a value of the actual data flow is already in my locally saved data (CSV)? I don't want to get rid of the duplicates. But I need to add a column which signalizes if the value was already seen or not. How is this possible?   Question 4 At last, I am struggling how to save my files, because I need them to be saved separately and additionally appended to a combined file. The separated files are basic and working fine. About appending to a file I read about different solutions, mostly about Groovy scripts.  Is ExecuteGroovyScripts the right way to go?    I hope you can help me, and I am looking forward to your answers.      Best regards   Maurice   ... View more