Member since
06-14-2023
87
Posts
27
Kudos Received
8
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1798 | 12-29-2023 09:36 AM | |
2008 | 12-28-2023 01:01 PM | |
594 | 12-27-2023 12:14 PM | |
302 | 12-08-2023 12:47 PM | |
1007 | 11-21-2023 10:56 PM |
08-28-2023
07:10 AM
The 2 lines I provided are the very first lines and then all your code comes after.
... View more
07-29-2023
07:56 PM
NiFi is awesome with so many out-of-the-box processors...however, I have found sometimes a very specialized scripted Groovy processor that fetches 1000 or more files at a time to perform significantly faster... especially if your custom processor consolidates several processors into 1.
... View more
07-29-2023
11:16 AM
2 Kudos
You should be able to create a custom scripted processor and define an attribute that's sensitive so once you set it...it won't be visible. You might be able to handle your whole process inside the script so the password never gets written out. I've written several in Groovy. An alternative option is to leverage something like Vault (Vault by HashiCorp (vaultproject.io)). PropertyDescriptor OAUTH_TOKEN = new PropertyDescriptor.Builder() .name("oAuth Token") .displayName("oAuth Token") .description("oAuth Token") .required(true) .sensitive(true) .addValidator(StandardValidators.NON_EMPTY_VALIDATOR) .build()
... View more
07-02-2023
12:15 PM
If you download the Postgres JDBC driver's you can leverage any of the SQL processors to query Postgres directly instead of using pgdump. https://jdbc.postgresql.org/download/
... View more
06-26-2023
06:51 AM
1 Kudo
Use an UpdateAttribute processor to add attributes with your header values. i.e. Content-Type with value of application/json and in the InvokeHTTP you'll notice there's a field called Attributes to Send (or something like this... They will be the headers) and list those Accept|Authorize|Any other (it's regular expression so you might have to escape some characters). FlowFile content should be sent as the body.
... View more
06-25-2023
12:23 PM
Have you tried these examples? https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-1/ta-p/248922
... View more
06-25-2023
12:21 PM
Apache PDFBox is supposed to allow you to merge PDF content. Since this is a Java library you can create a scripted Groovy processor to merge the files for you. https://pdfbox.apache.org https://javadoc.io/doc/org.apache.pdfbox/pdfbox/2.0.27/index.html
... View more
06-25-2023
11:28 AM
Have you seen if this has what you're referring to? https://community.cloudera.com/t5/Community-Articles/ExecuteScript-Cookbook-part-3/ta-p/
... View more
06-24-2023
08:16 PM
I've done this With several Java libraries and Groovy based InvokeScriptedProcessor. Just looked up Kazoo... You can do the same with Python/Jython as long as the Kazoo module is 100% Python and not C based
... View more
06-24-2023
05:55 PM
First, the incoming Data should be proper JSON that can be easily parsed and processed...i.e. something like this [
[
{
"ID": "100000",
"Date": "2022-09-22",
"Start Time": "08:00",
"End Time": "14:00"
},
{
"ID": "100001",
"Date": "2022-09-02",
"Start Time": "08:00",
"End Time": "14:00"
},
{
"ID": "100002",
"Date": "2022-09-02",
"Start Time": "08:00",
"End Time": "14:00"
}
],
{
"res_data": [
{
"Record": "Record 2",
"Description": "Invalid values for ID or Date"
},
{
"Record": "Record 3",
"Description": "Invalid values for ID or Date"
}
]
}
] Then Groovy code like this: import groovy.json.JsonOutput
import groovy.json.JsonSlurper
JsonSlurper jsonSlurper = new JsonSlurper()
JsonOutput jsonOutput = new JsonOutput()
List data = jsonSlurper.parseText('''
[
[
{
"ID": "100000",
"Date": "2022-09-22",
"Start Time": "08:00",
"End Time": "14:00"
},
{
"ID": "100001",
"Date": "2022-09-02",
"Start Time": "08:00",
"End Time": "14:00"
},
{
"ID": "100002",
"Date": "2022-09-02",
"Start Time": "08:00",
"End Time": "14:00"
}
],
{
"res_data": [
{
"Record": "Record 2",
"Description": "Invalid values for ID or Date"
},
{
"Record": "Record 3",
"Description": "Invalid values for ID or Date"
}
]
}
]
''')
println(data)
println("=" * 80)
List<Map<String, String>> transformedList = []
data[0].eachWithIndex { map, index ->
Map<String, Object> record = data[1]["res_data"].find { it["Record"] == "Record ${index + 1}" }
if (record) {
map.putAll(record)
}
transformedList.add(map)
}
println(jsonOutput.prettyPrint(jsonOutput.toJson(transformedList))) Creates an output like this: [[[ID:100000, Date:2022-09-22, Start Time:08:00, End Time:14:00], [ID:100001, Date:2022-09-02, Start Time:08:00, End Time:14:00], [ID:100002, Date:2022-09-02, Start Time:08:00, End Time:14:00]], [res_data:[[Record:Record 2, Description:Invalid values for ID or Date], [Record:Record 3, Description:Invalid values for ID or Date]]]]
================================================================================
[
{
"ID": "100000",
"Date": "2022-09-22",
"Start Time": "08:00",
"End Time": "14:00"
},
{
"ID": "100001",
"Date": "2022-09-02",
"Start Time": "08:00",
"End Time": "14:00",
"Record": "Record 2",
"Description": "Invalid values for ID or Date"
},
{
"ID": "100002",
"Date": "2022-09-02",
"Start Time": "08:00",
"End Time": "14:00",
"Record": "Record 3",
"Description": "Invalid values for ID or Date"
}
]
... View more