Member since
11-16-2015
905
Posts
666
Kudos Received
249
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 514 | 09-30-2025 05:23 AM | |
| 850 | 06-26-2025 01:21 PM | |
| 761 | 06-19-2025 02:48 PM | |
| 937 | 05-30-2025 01:53 PM | |
| 11701 | 02-22-2024 12:38 PM |
07-30-2018
09:00 PM
Oracle has different syntax for aliasing columns (i.e. use "AS") versus tables (i.e. don't use "AS"). The existing code in 1.7.0 hardcodes the "AS" keyword. I have written NIFI-5471 to delegate the generation of the table alias clause to the database adapter. Unfortunately I am not aware of any workaround.
... View more
07-30-2018
08:40 PM
In NiFi 1.7.0 I believe you can right-click on the processor and choose "Terminate threads". If for some reason that doesn't work I think you have to restart the NiFi instance.
... View more
07-30-2018
06:26 PM
ValidateRecord is more about validating the individual records than it is about validating the entire flow file. If some records are valid and some are invalid, each type will be routed to the corresponding relationship. However, for invalid records, we can't use the same record writer as valid records, or else we know it will fail (because we know they're invalid), so there is a second RecordWriter for invalid records (you might use this to try to record the field names or something, but by the time that ValidateRecord knows the individual record is invalid, it doesn't know that it came in as Avro (for example), nor does it know that you might want it to go out as Avro. That's the flexibility and power of the Record Reader/Writer paradigm, but in this case the tradeoff is that you can't currently treat the entire flow file as valid or invalid. It may make sense to have a "Invalid Record Strategy" property, to choose between "Individual Records" using the RecordWriters (the current behavior), or "Original FlowFile" which would ignore the RecordWriters and instead transfer the entire incoming flow file as-is to the 'invalid' relationship. Please feel free to file an improvement Jira for this capability.
... View more
07-30-2018
05:49 PM
1 Kudo
When you see the number in the upper-right hand corner, that refers to the fact that even though the processor is "stopped", there are still threads running. You won't be able to edit the configuration or restart it until those threads have stopped (the number and icon will disappear).
... View more
07-27-2018
06:38 PM
1 Kudo
What version of NiFi are you using? I tried this on the latest snapshot (to become NiFi 1.8.0) and using double-quotes in the Maximum Value Column worked for me (i.e. it does incremental fetching). Looking at the code it seems like it should work in NiFi 1.7.x as well, I'm wondering if there was a bug in your version that we've since fixed.
... View more
07-27-2018
06:15 PM
The column name itself has double quotes?
... View more
07-27-2018
05:53 PM
What issues are you having? That flow description seems like it should work. Perhaps your regular expression or other config of ExtractText needs tweaking?
... View more
07-26-2018
05:46 PM
1 Kudo
Currently the processor(s) do a simple "put" of the value into the attribute whose name is the same as the parameter key. Previous encounters with a parameter of the same name will be overwritten, which is why you're seeing . I have written up an Improvement Jira (NIFI-5467) to let the user choose to allow multiple params with the same key, and create a comma-separated list when multiple params are encountered. As a workaround you can parse the http.query.string using the same technique with which I answered your related question, using a script to parse the string and create an array of values for the multiple-valued param.
... View more
07-26-2018
05:02 PM
1 Kudo
AFAIK, Jolt doesn't have functions for doing transformations of a single value (such as split, extract, etc.). If the "params" field will always contain the same parameters in the same order (at least sn being first, c being second, and 4 p's) then you can extract "params" into an attribute using EvaluateJsonPath (with a JSONPath of $.params), then an update attribute with something like this: If the parameters are arbitrary (such as number of p's, or can be in any order), you might be better off with ExecuteScript for your custom logic. Here's a Groovy script you can use in ExecuteScript to do what you describe above: import org.apache.commons.io.IOUtils
import java.nio.charset.StandardCharsets
import groovy.json.*
def flowFile = session.get()
if(!flowFile) return
flowFile = session.write(flowFile,
{ inputStream, outputStream ->
def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
def obj = new JsonSlurper().parseText(text)
def params = obj.params.tokenize('&')
def builder = new groovy.json.JsonBuilder()
builder.call {
'queryType' 'scan'
'dataSource' 'xyz'
'resultFormat' ' list'
'columns' params.findAll {p -> p.tokenize('=')[0] == 'p'}.collect {p -> p.tokenize('=')[1]}
'intervals' ([] << '2018-01-01/2018-02-09')
'filter' {
'type' 'selector'
'dimension' 'sn'
'value' params.find {p -> p.tokenize('=')[0] == 'sn'}.tokenize('=')[1]
}
}
outputStream.write(builder.toPrettyString().getBytes(StandardCharsets.UTF_8))
} as StreamCallback)
flowFile = session.putAttribute(flowFile, "filename", flowFile.getAttribute('filename').tokenize('.')[0]+'_translated.json')
session.transfer(flowFile, ExecuteScript.REL_SUCCESS)
... View more
07-26-2018
02:38 AM
1 Kudo
For PartitionRecord, you'll want a "RecordPath" that points to the field in the schema that holds the table code value. Assuming it's called "tableCode", the RecordPath would be /tableCode
... View more