Member since
10-24-2016
47
Posts
16
Kudos Received
0
Solutions
01-04-2017
09:27 AM
I see an option 'Concurrent Tasks' in Scheduling in Nifi version 2.0. Here are my questions regarding Concurrency. 1. I see this only at indivudual processor level. Can this be set at 'Processor Group' level. In short can I parameterize it. 2. How can I arrive at this number. What are all the fators that decide concurrency. E.g. Memory, average data load etc, Other application in the same box like spark.
... View more
Labels:
- Labels:
-
Apache NiFi
11-15-2016
10:26 AM
I have a CSV file with 9 columns. How can I remove duplicates among columns 4 through 9? What we tried: 1. Split 1-4 columns in a file 2. Split 4-9 columns -> Deduplicate records Now, i tried using 'ReplaceTextWithMapping' to merge the files with 4th column (Common on both files). But I am not sure if my approach is right. Is there any other way to achieve this.
... View more
Labels:
- Labels:
-
Apache NiFi
11-15-2016
07:16 AM
I am using the CSVToAVRO converter processor. Is there a way I can mention the field as "not null" in the record schema. Or, do we have any other method to do this
... View more
Labels:
- Labels:
-
Apache NiFi
11-14-2016
02:03 PM
I am using the 'ConvertCSVToAVRO' processor. My requirement is, I should further process the file only if all the records matches the schema. But, now, the processor generates flowfile as soon as the 'record' matches the schema. How can i continue with the flow only with 'entire file' matches the schema
... View more
Labels:
- Labels:
-
Apache NiFi
11-10-2016
01:19 PM
1 Kudo
I have a requirement where I will have to decide the flow based on the record header. Since the file is huge, I don't want to waste time reading the entire file where only one record is enough. Is there a way I can achieve this.
... View more
Labels:
- Labels:
-
Apache NiFi
11-10-2016
10:33 AM
3 Kudos
I have a 'Control A' separated file. I am trying to append each line with a date value with the regular expression, ${srcdelim}${now():format('mm-dd-yy')} where ${srcdelim} contains the value '\u0001' from a schema file. Apparently, the output file contains the string \u0001 rather than a control A delimiter. I have attached my sample input and output. Character Set: UTF-8
... View more
Labels:
- Labels:
-
Apache NiFi
11-09-2016
01:33 AM
I am trying to Append each line with a time-stamp or a date like ${srcdelimiter}${now():format('mm-dd-yy')}
... View more
11-08-2016
03:23 PM
@Matt Burgess we decided to do a routeOnAttribute for files which are separated by Control A or CSV. We now face a new challenge while using Control A separated file. What we do is, we get this delimiter as a flowFile attribute and we use this attribute say ${srcdelim} (whose value is - \u0001) in ReplaceText Processor. The resultant file is not Control A separated file. I just see a character \u0001 instead. can you please help me replace a control A character
... View more
11-08-2016
09:34 AM
2 Kudos
Can someone please provide me with some link that explains the basic concepts of Apache Nifi, Flowfile, how efficiently it handles memory, master slave relations, etc?
... View more
Labels:
- Labels:
-
Apache NiFi
11-03-2016
05:45 AM
Thanks @Matt Burgess and @Andy LoPresto. My requirement here is, I am keeping watch on a folder where my input files come in. These files can have different delimiters \u0001, \u0002 or a CSV. So, I was thinking if we can convert these files into AVRO by passing the delimiter as an attribute in the flowfile. Is there an efficient way to handle this other than using replace text.
... View more
- « Previous
-
- 1
- 2
- Next »