Member since
07-30-2019
333
Posts
356
Kudos Received
76
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
9859 | 02-17-2017 10:58 PM | |
2294 | 02-16-2017 07:55 PM | |
7974 | 12-21-2016 06:24 PM | |
1757 | 12-20-2016 01:29 PM | |
1232 | 12-16-2016 01:21 PM |
02-17-2017
10:58 PM
1 Kudo
GetFile will delete the processed file by default. Check the Keep Source File setting, for example. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.GetFile/index.html A better alternative for processing files without touching them is a combination of ListFile/FetchFile processors. These will maintain the state internally in NiFi to track the processed files. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ListFile/index.html
... View more
02-16-2017
07:55 PM
3 Kudos
Hi Jobin, The backpressure should be treated as a threshold, not an absolute barrier. Due to scheduling semantics and some processors like e.g. UpdateAttribute doing internal micro-batching, the number of objects (or size) can go over the set limit. It will still engage the backpressure and will not turn it off until the level falls below the threshold, but in general you shouldn't worry too much about the number going slightly above. This is merely due to internal implementation details.
... View more
01-30-2017
05:12 PM
1 Kudo
Tim, It looks like the regex you're using might need more work to account for optional matches. The ReplaceText processor is complaining about missing match groups and has no context to do the replacement.
... View more
01-03-2017
04:06 PM
Make sure you split data using the SplitJson processor in NiFi before putting into Splunk. The reason is the syslog receiver may bundle incoming messages based on the network setup, but knows nothing about actual data format like json.
... View more
12-21-2016
06:49 PM
1 Kudo
You removed all details from the configuration. Do you have the 'temp' user? Which user do you use for SFTP login? Normally you wouldn't have access to other users' home directories. Try using a directory outside of /home and set correct permissions on it.
... View more
12-21-2016
06:24 PM
2 Kudos
Setting a remote file to /home/temp/ is wrong. It expects a filename, not a directory. A combination of ListSFTP and FetchSFTP will ensure the filenames are passed to the FestSFTP via a known attribute name. Here's an example template from the gallery: https://github.com/hortonworks-gallery/nifi-templates/blob/master/templates/List_and_Fetch_SFTP_template.xml
... View more
12-20-2016
01:29 PM
2 Kudos
Yes, use MonitorActivity processor: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.MonitorActivity/index.html
... View more
12-16-2016
01:21 PM
1 Kudo
One needs to allow for a reflect function, it's blacklisted by default. See e.g. this discussion: https://community.hortonworks.com/questions/25828/udf-reflect-is-not-allowed-beeline.html
... View more
12-15-2016
06:52 PM
1 Kudo
ExecuteProcess doesn't take input. It is a source of data (e.g. from that process). What you're looking for is ExecuteStreamCommand, which allows for passing in a flowfile and evaluate attributes dynamically: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ExecuteStreamCommand/index.html
... View more
12-13-2016
08:51 PM
3 Kudos
David, it's here: http://docs.hortonworks.com/HDPDocuments/HDF2/HDF-2.1.0/bk_dataflow-command-line-installation/content/hdf_isg_hardware.html Although I don't think guidelines have changed.
... View more