Member since
07-30-2019
898
Posts
195
Kudos Received
91
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
704 | 10-05-2021 01:53 PM | |
12935 | 09-23-2019 06:03 AM | |
4836 | 05-04-2019 08:42 PM | |
912 | 06-11-2018 12:45 PM | |
8946 | 06-04-2018 01:11 PM |
07-28-2017
06:50 PM
1 Kudo
@Quentin T Here is a link to a tutorial that will help show how these tools work together and what role they play. REAL-TIME EVENT PROCESSING IN NIFI, SAM, SCHEMA REGISTRY AND SUPERSET
... View more
07-28-2017
03:01 PM
1 Kudo
@Foivos A You're seeing that error because there is a blank space in the file name, try doing this: ${filename: prepend(${now():format("yyyy-MM-dd-HH:mm:ss")})}
... View more
07-27-2017
09:09 PM
@Bilel Boubakri I am assuming you know how many subdirectories there are in this solution. ListFTP - scanning the main directory with all the subdirectories EexcutScript - running a groovy script waiting for all of the flag.ok files to be in the subdirectories FetchFTP - configured to pull the wanted files Here is a snapshot of the simple flow: Here is the sample groovy script for the ExecuteScript processor, configured for this example to wait for 5 files: def flowFiles = session.get(5)
if(!flowFiles || flowFiles.size() < 5) {
session.rollback()
} else {
session.transfer(flowFiles, REL_SUCCESS)
}
waiting-for-5-files-before-ftping-the-files.xml Here is a template of the sample flow
... View more
07-27-2017
06:07 PM
@Hadoop User
100 concurrent tasks, unless you have a large number of CPUs available, is too many. Try using 4 concurrent tasks and a run duration of 2 seconds. How long does that take to process the 50k messages? What does the RouteOnContent processor configuration look like?
... View more
07-26-2017
11:54 PM
1 Kudo
@Gayathri You might be encountering this issue Issue with the MergeContent Processor when processing Avro files
... View more
07-26-2017
11:44 PM
@Hadoop User
How many and what are the size of typical files from the SplitContent processor? How many Concurrent tasks does the RouteOnContent processor have configured? Try increasing the Run Duration and/or Concurrent Tasks for better throughput.
... View more
07-25-2017
12:00 AM
@Bilel Boubakri Are you saying you want to wait until each sub-directory has a specific file and then start ftping the files to a local directory?
... View more
07-19-2017
07:54 PM
@Jobin George The fix is being worked currently so you don't have to copy over the flow.xml.gz manually.
... View more
07-19-2017
07:43 PM
1 Kudo
@bpreachuk If NiFi is unsecured, there will not be a users.xml and authorizations.xml files.
... View more
07-19-2017
06:23 PM
@Jobin George That is a step I have always had to do when adding a new node to an existing cluster when using Ambari
... View more