Member since
04-11-2016
471
Posts
325
Kudos Received
118
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2078 | 03-09-2018 05:31 PM | |
2655 | 03-07-2018 09:45 AM | |
2539 | 03-07-2018 09:31 AM | |
4408 | 03-03-2018 01:37 PM | |
2476 | 10-17-2017 02:15 PM |
06-16-2016
08:03 AM
Hi @Chris Mazo, I am not aware of a processor to deal with ISO8583 messages. I think that your best options are: write your own custom processor if you want to deal with some custom specifications use ExecuteScript processor and write your code in a language like Groovy directly in NiFi (you have some great examples here: http://funnifi.blogspot.fr/) Hope this helps.
... View more
06-15-2016
06:20 AM
2 Kudos
Hi @awen hu
I'd use the ExtractText processor:
Evaluates one or more Regular Expressions against the content of a
FlowFile. The results of those Regular Expressions are assigned to
FlowFile Attributes. Regular Expressions are entered by adding
user-defined properties; the name of the property maps to the Attribute
Name into which the result will be placed. The first capture group, if
any found, will be placed into that attribute name.But all capture
groups, including the matching string sequence itself will also be
provided at that attribute name with an index value provided, with the
exception of a capturing group that is optional and does not match - for
example, given the attribute name "regex" and expression "abc(def)?(g)"
we would add an attribute "regex.1" with a value of "def" if the "def"
matched. If the "def" did not match, no attribute named "regex.1" would
be added but an attribute named "regex.2" with a value of "g" will be
added regardless.The value of the property must be a valid Regular
Expressions with one or more capturing groups. If the Regular
Expression matches more than once, only the first match will be used.
If any provided Regular Expression matches, the FlowFile(s) will be
routed to 'matched'. If no provided Regular Expression matches, the
FlowFile will be routed to 'unmatched' and no attributes will be applied
to the FlowFile. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ExtractText/index.html Hope this helps.
... View more
06-14-2016
08:19 AM
Let's say my files contain the date in its name: /tmp/test/test-20160501.txt ... /tmp/test/test-20160531.txt /tmp/test/test-20160601.txt ... /tmp/test/test-20160614.txt If I only want to list files for the month of June: Note: when hovering the "?" symbol next to a property in processor settings you are able to see if the property supports expression language.
... View more
06-14-2016
06:53 AM
4 Kudos
Hi @Manikandan Durairaj, If you don't have dates in the name of your files, the only option I see is to use the combination of ListFile and FetchFile processors. The ListFile processor has the two following properties: Minimum File Age The minimum age that a file must be in order to be pulled; any file younger than this amount of time (according to last modification date) will be ignored
Maximum File Age The maximum age that a file must be in order to be pulled; any file older than this amount of time (according to last modification date) will be ignored It will allow you to get files with the last modification dates in the given interval. By connecting the output of this processor to the FetchFile processor you will get the files just like the GetFile processor does. If this is not what you are looking for, we need more details about what you want to achieve. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ListFile/index.html Hope this helps.
... View more
06-13-2016
07:45 PM
2 Kudos
Hi @Todd Wilkinson, I'd use an UpdateAttribute processor by adding two dynamic properties and using expression language as follow: attribute2 => ${attribute1:substringBefore('T')} attribute3 => ${attribute1:substringAfter('T')} Hope this helps. Expression language guide : https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html
... View more
06-13-2016
11:46 AM
Hi @Benjamin Leonhardi, thanks for your answer, it confirms what I read so far. I'll go that way, thanks again.
... View more
06-13-2016
09:56 AM
Hi, What is the best way to have an Oozie action performing a multiple tables export using Sqoop? Context: I have a HDFS directory containing one file per destination table but the number of files is not constant over time The filename allows me to know what is the name of the target table I'd like to have one single Oozie action calling Sqoop to export all the files in the target database. Is it something we can do? What are the available options?
... View more
Labels:
- Labels:
-
Apache Oozie
-
Apache Sqoop
06-09-2016
11:24 AM
In the case I want to export data, using Sqoop, from HDFS to an external destination (Teradata for example), is there a recommendation regarding the format of the input files? AFAIK, supported formats are : Delimited text files Sequence files ORC files Do we observe performance differences between input formats? Thanks
... View more
Labels:
- Labels:
-
Apache Sqoop
06-09-2016
09:35 AM
Hi @KC Any information in the application logs (./logs/nifi-app.log) ?
... View more
06-07-2016
12:53 PM
This works for me.
... View more