Member since
09-29-2015
871
Posts
723
Kudos Received
255
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3416 | 12-03-2018 02:26 PM | |
2367 | 10-16-2018 01:37 PM | |
3680 | 10-03-2018 06:34 PM | |
2452 | 09-05-2018 07:44 PM | |
1862 | 09-05-2018 07:31 PM |
08-23-2016
01:14 PM
Ah right, I'm used to setting this up with ListHDFS + FetchHDFS which is different because its a shared resource... You are right that it is not going to work correctly when it is not a shared location because another node can't fetch a file that is only on primary node. Sorry about the confusion.
... View more
08-22-2016
01:59 PM
1 Kudo
This is the correct setup... you would likely want to schedule ListFile to run on Primary Node only, this is an option on the scheduling tab of the processors. That would take care of the problem when the directory is not a shared location, but even when it is shared, you still probably want Primary Node only because you don't want two instances of ListFile retrieving the same files.
... View more
08-20-2016
12:38 AM
Can you try setting nifi.remote.input.socket.host=localhost on the one where 9091 is set, you'll have to restart after changing that value. Curious to see if that changes behavior.
... View more
08-18-2016
11:14 PM
1 Kudo
Tunneling 9091 to 9091 seems correct, I'm wondering if the hostname is also a problem... did you also set a value for nifi.remote.input.socket.host? I have a feeling that its trying to connect to the hostname or ip, rather than localhost:9091.
... View more
08-18-2016
09:10 PM
On the instance where you have POV-HDFSInput, what is the value of nifi.remote.input.socket.port in nifi.properties? You would need to be tunneled as well, and i think it would have to be tunneled to the same port, since the first NiFi is going to make a REST call http://localhost:10090/nifi and ask what port to use for site-to-site and it is going to return the value of nifi.remote.input.socket.port.
... View more
08-17-2016
05:04 PM
Can you provide an example JSON document that is not splitting correctly?
... View more
08-17-2016
05:03 PM
5 Kudos
1) You can not apply EL to the content of a flow file, this is by design, The approach you mentioned with ExtractText is correct. 2) You might be able to do this with a custom processor, or possibly somehow using the distributed cache services, but in general a processor is operating on one flow file at a time.
... View more
08-17-2016
05:01 PM
5 Kudos
The ExtractText processor is used to pull up text from the content of a flow file into an attribute. You should be able to add a property like "message.body" with a value of "(.*)" to ExtractText to get it to extract everything. Keep in mind this only makes sense for small messages. Attributes are held in memory and if you did this with a huge flow file you could blow up the memory of your Java process.
... View more
08-17-2016
01:43 AM
1 Kudo
@Randy Gelhausen NiFi JIRA to capture this idea: https://issues.apache.org/jira/browse/NIFI-2585
... View more
08-16-2016
06:31 PM
1 Kudo
What do you think about having it automatically add two attributes like "remote.host" and "remote.address" where remote.host has just the hostname and remote.address has hostname:port?
... View more