Member since
09-29-2015
58
Posts
76
Kudos Received
17
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2090 | 01-25-2017 09:19 PM | |
2984 | 11-02-2016 02:54 PM | |
3013 | 09-08-2016 01:36 AM | |
5137 | 08-09-2016 07:52 PM | |
1360 | 06-30-2016 06:09 PM |
06-30-2016
06:09 PM
3 Kudos
@Timothy Spann You should use the MaxMindDB format. I've always used the City database, located at http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz
... View more
06-29-2016
06:29 PM
Ah, thanks for the background! Whether or not you will have issues on fetching, replacing, and putting will depend on how big the file is, and how often this logic is happening. If it is 1200 iteration each time the logic runs and the logic runs every 24 hours then you will be fine. If it runs every 1 second then you may be hurting with this solution 🙂 A possibly better approach may be to use the ExecuteScript processor and write a simple Groovy or Python script that updates the file for you. Or you could potentially look at implementing some logic that would allow you to update the file only once at the end, if possible, instead of every one of the 1200 iterations, etc.
... View more
06-29-2016
05:13 PM
@mliem I think I misunderstood a little bit what you are trying to do. I thought you were trying to just append something to the end of the file each time. But it looks like you're actually trying to replace the value. So instead of appending to the file, in ReplaceText you can use the Regex Replace "Replacement Strategy." For example, the Search Value may be something like: (${count} ${TagName}),.* with a Replacement Value of: $1,${now():format('yyyy-MM-dd HH:mm:ss')} So if the 'count' attribute is 3 and the 'TagName' attribute is 'tag3' you would replace '3 tag3,2016-01-03 13:01:01' with '3 tag3,2016-06-29 13:16:33:50' for instance. Is this a little more along the lines of what you're looking for?
... View more
06-29-2016
04:28 PM
1 Kudo
@mliem you are correct in that it would replace the whole file. One option for you may be to use FetchFile first to retrieve the file of interest. Then use ReplaceText to append text to the end of it, and then use PutFile to overwrite the contents of the file, with the updated content.
... View more
06-22-2016
01:23 PM
1 Kudo
@Sai Dileep that log message indicates that the NiFi process has started, and it is listening for "shutdown" commands, etc. from the bootstrap. However, it does not indicate that the dataflow has started or that the web UI is ready. It will typically take 1-2 minutes to startup. Do you have anything else in the nifi-app.log file? If not, can you check the nifi-bootstrap.log file and see if there is something going on there? Additionally, in the conf/bootstrap.conf file, please make sure that you don't have the process setup to listen for a Remote Debugger. That would be done by looking for this line around line 39 in the file: #java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000 If you see that the above line is un-commented and that suspend=y, that could cause the process to not finish starting.
... View more
06-16-2016
09:07 PM
6 Kudos
@Timothy Spann there is a JIRA created for bringing Calcite into a NiFi processor - https://issues.apache.org/jira/browse/NIFI-1280. The idea originally was to use it to filter out specific columns in incoming CSV data. However, as we looked at it, we found that there is a lot more that this can do! Initially, it will likely be used to simply run SQL over CSV data, with each incoming FlowFile being transformed into an outgoing FlowFile. Eventually, I would like to see additional data formats being introduced into this, so that SQL could be run over any number of different data formats to filter, transform, etc.
... View more
02-23-2016
06:22 PM
2 Kudos
The PutEMail processor does not send the contents of a FlowFile by default. Rather, it sends the message that is set in the "Message" property. This is done because quite often, we want to send an email notification that something happened but not send the contents of the FlowFile itself. You can, however, set the "Attach File" property to true, which will send the contents of the FlowFile as an attachment to the e-mail.
... View more
02-03-2016
04:10 PM
2 Kudos
It looks like it cannot resolve the hostname 'provaha' to an IP address - can you check if the provaha is in your /etc/hosts file, or supply the FQDN instead?
... View more
01-27-2016
03:41 PM
2 Kudos
@michelsumbul the GetSFTP processor does not allow you to specify the host dynamically. It is intended to continually poll a specific host and pull the data into NiFi, then delete the original file from the host. Often, in the Open Source world, it's important that we not delete the data from the source, and so we have moved more toward using the ListSFTP and FetchSFTP processors. The FetchSFTP processor does allow you specify the hostname and directory dynamically, so you could perform a query and then add FlowFile attributes from that query and send those FlowFiles to the FetchSFTP processor instead of GetSFTP.
... View more
01-21-2016
07:32 PM
1 Kudo
You may also want to consider that JMS messages tend to be quite small, and before pushing to HDFS it may help to merge together many JMS messages so that you don't have a huge number of files in HDFS. You could use the MergeContent Processor to accomplish this.
... View more