Member since
09-11-2015
60
Posts
72
Kudos Received
17
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1752 | 04-11-2017 06:18 PM | |
1405 | 02-21-2017 03:04 PM | |
1993 | 12-28-2016 10:21 PM | |
4492 | 11-30-2016 04:17 AM | |
6563 | 11-08-2016 02:42 PM |
01-18-2016
04:06 PM
1 Kudo
Currently TailFile only works on a systems local system. In order to coordinate receiving data from a remote host there is numerous different options and configurations that need to be taken into account. Tail File was designed to fulfill a targeted goal, tail a local file. This is done to create a smaller building block which can build up the bigger system and is easier to re-use for multiple different applications. Getting log file data off a remote host with all the perks of NiFi is something we are working toward with a sub-project tentatively called "MiNiFi". There is a design proposal requesting feedback here: "http://apache-nifi-developer-list.39713.n7.nabble.com/DISCUSS-Proposal-for-an-Apache-NiFi-sub-project-MiNiFi-td6141.html". Please reply with any comments or suggestions!
... View more
01-14-2016
06:09 PM
What's your end goal JSON schema look like? What identifiers are each of the values going to use (for every line)? As a note, AttributesToJSON only creates flat JSON objects (no nested fields).
... View more
01-14-2016
05:19 PM
Ah so you also want to extract the text to be attributes of the flowfile. Is the structure of the contents only ever two lines and do you want to create JSON using both of those lines or split them into separate flowfiles?
... View more
01-14-2016
04:57 PM
1 Kudo
This is a bit tricky and will require a bit of regex magic. You will want to capture each length of data attribute (2, 4 and 2 respectively) into capture groups then use those capture groups to replace the content. You'll use the ReplaceText processor with a search value of "(.{2})(.{4})(.{2})" and a replacement value of "$1;$2;$3" and configure it to evaluate line by line. This will go through the contents of the flowfile line by line and replace the contents like you want. Comment below if you run into any problems!
... View more
01-13-2016
10:06 PM
The GetFile processor gets most of the file attributes when it ingests the file (see image). Is there specific attributes that you don't see in your FlowFile being produced by GetFile that you need? The ExtractImageMetadata processor is solely for extracting metadata that is formatted into the image file itself and won't help to get system level file metadata.
... View more
01-11-2016
07:26 PM
You shouldn't need to configure the ExtractImageMetadata processor aside from the max number attributes to add to the flowfile (so you don't blow away your java heap). It will automatically extract all the metadata it can from the supported formats. PNG and JPEG are supported file formats so if it's not extracting certain metadata from those then the metadata format may not be supported. Do you know the metadata format you're using? Extracting metadata from PDF files is not currently supported. If we were to add a new processor, what functionality and requirements would you like for it?
... View more
01-11-2016
03:02 PM
2 Kudos
The ExtractImageMetadata processor uses this image metadata extraction library: https://github.com/drewnoakes/metadata-extractor It is able to process many formats of metadata and types of files including JPEG and PNG. What format is the metadata on your JPEG and PNG files that isn't getting properly extracted? Unfortunately it does not process PDF format files but that could be a good new processor to have. What functionality and requirements would you like for a ExtractPdfMetadata processor?
... View more
12-28-2015
12:57 AM
Andrew is correct in regards to the difference between ${uuid} and ${UUID()} but the reason why ${uuid} doesn't work is because the GetHTTP processor doesn't evaluate the expression language in relation to the new flowfile.
As you can see on this line: https://github.com/apache/nifi/blob/528dab78d6dbc3bb4f61663f423b07284936ec40/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GetHTTP.java#L476
It calls "evaluateAttributeExpressions()" with no parameters so the "uuid" of the new flowfile is outside the scope of the expression language when being evaluated. If you want to re-use the UUID for the FlowFile you can use an UpdateAttribute processor to append the UUID to "./downloads/".
... View more
10-30-2015
12:51 AM
1 Kudo
What you're looking for is a patch that was recently merged into master:https://issues.apache.org/jira/browse/NIFI-447 It allows the ReplaceText processor to do multiple new things such as prepend or append a property (expression language supported) to either every line or the whole flowfile.
... View more
10-13-2015
09:24 PM
1 Kudo
1: The ports needed are the UI ports (http and/or https) and any ports needed by processors for data. The two config properties are: nifi.web.http.port and nifi.web.https.port. 2: Unicasting is used by default for the clustered environment and multicasting isn't needed. Multicasting is used to facilitate broadcasting the availability of a NiFi NCM service. Though there are some network topologies that disable multicasting. So all the multicasting properties are blank since they aren't used by default. If you're looking for a good guide to setting up a simple cluster you can find one here: https://kisstechdocs.wordpress.com/2015/01/15/simple-tasks-in-nifi-creating-a-super-simple-cluster/ 3: Site-to-site is a bit tricky with the word "port". When talking about NiFi only, there are "Input Ports" and "Output Ports" for "Remote Processes Groups" (RPG). All of the RPG's input ports are fed via the network port set with this NiFi system property "nifi.remote.input.socket.port". For connections leaving a NiFi instance, whether thats via a RPG's output port or a processor like HttpGet, a random port that's available will be allocated by the system and the connection kept open. It will be closed after a period of inactivity.
... View more
- « Previous
- Next »