Member since
11-16-2015
905
Posts
665
Kudos Received
249
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 416 | 09-30-2025 05:23 AM | |
| 735 | 06-26-2025 01:21 PM | |
| 631 | 06-19-2025 02:48 PM | |
| 837 | 05-30-2025 01:53 PM | |
| 11331 | 02-22-2024 12:38 PM |
11-06-2017
04:39 PM
1 Kudo
You can extract values from JSON content into attributes using EvaluateJsonPath, then you can use RouteOnAttribute to do routing.
... View more
11-02-2017
01:57 PM
1 Kudo
ExecuteSQL will fetch and send all rows in the ResultSet each time it runs. If you don't want that many rows in a single flow file (but still want it to only execute once), use QueryDatabaseTable with no Max Value Columns set. This acts like ExecuteSQL if no Max-Value Columns are supplied (so you will still want to start/stop it), but also has options like Max Rows Per Flow File, etc.
... View more
11-01-2017
11:27 PM
1 Kudo
You can start and immediately stop the processor, it will be guaranteed to run at least once (set the Run Schedule to something like 10 seconds so it gives you enough time to start and stop it).
... View more
11-01-2017
01:13 PM
This feature was added in NiFi 1.4.0 (NIFI-4257)
... View more
10-31-2017
01:49 PM
The join() function, from the documentation, "may be used only in conjunction with the allAttributes , allMatchingAttributes , and allDelineatedValues functions". I think you want the append() function: temp_${now():format("yyyy-MM-dd-HH-mm-ss"):append(${random():mod(10):plus(1)})} I tested this with my EL tester and it seems to work. However you might want to append a dash or underscore before the random digit, as the above expression will put the digit exactly after the number of seconds, unless that's what you want.
... View more
10-30-2017
06:23 PM
1 Kudo
In later versions of NiFi, you may also consider using the "record-aware" processors and their associated Record Readers/Writers, these were developed to avoid this multiple-split problem as well as the volume of associated provenance generated by each split flow file in the flow.
... View more
10-30-2017
06:20 PM
1 Kudo
According to this, it looks like Teradata doesn't support the DOUBLE data type, and instead supports FLOAT as the JDBC type yet returns a Double object when getObject is called. When the Double is inserted into an Avro record whose field type is "float", you get the above error. This IMO is an issue with Teradata being out of spec with JDBC. Although NiFi could demote a Double into a float field, that is not prudent as there can be data/precision loss or other issues of which the user would never be made aware. A workaround might be to use a DECIMAL column instead of DOUBLE, or to cast the column(s) as @Shu suggested.
... View more
10-30-2017
05:05 PM
I don't currently see a processor that would do this for you, but if you are comfortable with a scripting language such as Groovy, Javascript, or Clojure, you could use ExecuteScript with any of these libraries to infer the character set for an incoming stream. For more information on including these library JAR(s) in your ExecuteScript configuration, see my ExecuteScript Cookbook (part 3) and/or my separate blog post. Since this seems like a good feature to have in NiFi proper, I have created a Jira (NIFI-4550) to add an InferCharacterSet processor.
... View more
10-26-2017
05:04 PM
2 Kudos
"Referencing Components" is done by the framework when you explicitly refer to a Component from another, such as selecting that DBCPConnectionPool from a drop-down list in PutSQL for example. With a scripting processor, the framework does not know what the script is doing, including whether it references a particular component or not. Also DBCPConnectionPool does not log that it executes the Validation Query, that is performed by Apache DBCP "under the hood" when a connection is requested from the pool. What is very weird about your situation is that the Controller Service (DBCPConnectionPool) becomes disabled, that should not be related to whether the connection pool gives back good/bad connections. Is there an error bulletin or something in the logs from the DBCPConnectionPool itself or the framework (not the ExecuteScript)?
... View more
10-25-2017
08:57 PM
6 Kudos
Some NiFi Expression Language (EL) expressions can be fairly complex, or used in a large flow, or both. These can make it difficult to test an EL expression on a running NiFi system. Although an excellent feature of NiFi is being able to adapt the flow while the system is running, it may not be prudent to stop a downstream processor, reroute a connection to something like UpdateAttribute, then list the queue in order to see attributes, content, etc. To make EL testing easier, I wrote a Groovy script called testEL.groovy that uses the same EL library that NiFi does, so all functions present in the specified NiFi version are available to the test tool. The following is the usage: usage: groovy testEL.groovy [options] [expressions]
Options:
-D <attribute=value> set value for given attribute
-help print this message As an example, the following tests an expression that appends "_world" to the "filename" attribute: > groovy testEL.groovy -D filename=hello '${filename:append("_world")}'
hello_world Note that it accepts multiple attribute definitions and multiple expressions, so you can test more than one expression using a single set of attributes: > groovy testEL.groovy -D filename=hello -D size=10 '${filename:append("_world")}' '${filename:prepend("I say "):append(" ${size} times")}'
hello_world
I say hello 10 times In order to attach testelgroovy.txt to this post, I had to add a .txt extension (and it lowercased the name), simply rename it before running the above. Hopefully you find this script helpful, if you try it please let me know how/if it works for you, and as always I welcome any questions, comments and suggestions on how to make things better 🙂 Cheers!
... View more
Labels: