2225
Posts
236
Kudos Received
83
Solutions
About
I'm not a product expert as my expertise is in online communities, and Knowledge systems but I try to provide links , ask clarifying questions or make connections to be helpful. Interests include: photography, travel, movies and consumer technology.
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1152 | 05-07-2025 11:41 AM | |
| 11740 | 12-04-2023 09:38 AM | |
| 4246 | 06-29-2023 05:42 AM | |
| 3642 | 05-22-2023 07:03 AM | |
| 2510 | 05-22-2023 05:42 AM |
06-15-2023
11:41 AM
This simple InvokeScriptedProcessor will look for a FlowFile attribute called "ip_address" and will attempt the reverse lookup and create a new attribute called "host_name" with the resolved value. import groovy.json.JsonOutput
import groovy.json.JsonSlurper
import java.net.InetAddress
import java.net.UnknownHostException
import java.nio.charset.StandardCharsets
import org.apache.commons.io.IOUtils
class GroovyProcessor implements Processor {
PropertyDescriptor BATCH_SIZE = new PropertyDescriptor.Builder()
.name("BATCH_SIZE")
.displayName("Batch Size")
.description("The number of incoming FlowFiles to process in a single execution of this processor.")
.required(true)
.defaultValue("1000")
.addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR)
.build()
Relationship REL_SUCCESS = new Relationship.Builder()
.name("success")
.description('FlowFiles that were successfully processed are routed here')
.build()
Relationship REL_FAILURE = new Relationship.Builder()
.name("failure")
.description('FlowFiles that were not successfully processed are routed here')
.build()
ComponentLog log
void initialize(ProcessorInitializationContext context) { log = context.logger }
Set<Relationship> getRelationships() { return [REL_FAILURE, REL_SUCCESS] as Set }
Collection<ValidationResult> validate(ValidationContext context) { null }
PropertyDescriptor getPropertyDescriptor(String name) { null }
void onPropertyModified(PropertyDescriptor descriptor, String oldValue, String newValue) { }
List<PropertyDescriptor> getPropertyDescriptors() { Collections.unmodifiableList([BATCH_SIZE]) as List<PropertyDescriptor> }
String getIdentifier() { null }
JsonSlurper jsonSlurper = new JsonSlurper()
JsonOutput jsonOutput = new JsonOutput()
def reverseDnsLookup(String ipAddress) {
try {
InetAddress inetAddress = InetAddress.getByName(ipAddress)
String hostName = inetAddress.getCanonicalHostName()
return hostName
} catch (UnknownHostException e) {
return "Unknown"
}
}
void onTrigger(ProcessContext context, ProcessSessionFactory sessionFactory) throws ProcessException {
ProcessSession session = sessionFactory.createSession()
try {
List<FlowFile> flowFiles = session.get(context.getProperty(BATCH_SIZE).asInteger())
if (!flowFiles) return
Map customAttributes = [:]
flowFiles.each { flowFile ->
String ipAddress = flowFile.getAttribute("ip_address")
if (ipAddress) {
String hostName = reverseDnsLookup(ipAddress)
customAttributes["host_name"] = hostName
flowFile = session.putAllAttributes(flowFile, customAttributes)
}
session.transfer(flowFile, REL_SUCCESS)
}
session.commit()
} catch (final Throwable t) {
log.error('{} failed to process due to {}; rolling back session', [this, t] as Object[])
session.rollback(true)
throw t
}
}
}
processor = new GroovyProcessor()
... View more
06-14-2023
02:33 PM
Does ExecuteSQL erase some of the attributes that could be used to associate the FlowFiles futher down stream?
... View more
06-09-2023
10:24 AM
@naveenb Your query will get better visibility by starting a new question in the community rather then asking on an already solved question. NiFi's ListSFTP and GetSFTP (deprecated in favor of listSFTP and FetchSFTP) processor only lists/gets files. When it generates a NiFi FlowFile from a file it finds recursively within the source SFTP server configured base directory, it adds a "path" attribute to that FlowFile. That "path" attribute has the absolute path to the file. So based on your configuration, the results you are seeing are expected since you configured your putSFTP with "/home/ubuntu/samplenifi/${path}" Were "path" attribute on your FlowFiles resolves to "/home/nifiuser/nifitest/sample" for files found in that source subdirectory. You can use NiFi expression language (NEL) to modify that "path" attribute string to get rid of the "/home/nifiuser" portion /home/ubuntu/samplenifi/${path:substringAfter('/home/nifiuser')} If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
05-26-2023
12:29 PM
@Gutao When interacting with the NiFi rest-api, I'd recommend creating a client certificate to use in your automation. A secured NiFi will always WANT a client certificate and will only try another configured auth method if a client certificate is not provide in the TLS exchange. Using a certificate for your rest-api automation removes the need for obtaining a token completely. You simply pass your client certificate with every rest-api call. Another advantage here over auth is token expiration. With no token involved with certificate based auth, your certificate will continuously work until it expires (typical default is 1 or 2 years). You'll need to setup authorization policies for your certificate user (Certificate DN used as user identity) for the various endpoints you are trying to interact with through the rest-api. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
05-24-2023
07:31 AM
Hi @ChuckE, yes i had seen that note before and imagine that they fixed a similar issue but didnt catch this specific case, i am posting an issue in Jira to hopefully find a fix.
... View more
05-23-2023
07:13 PM
Thanks for the reply. In my case the question is: I build a model and then I train the model again; How do I create a new build of the model with the same id of the model already created before? I have to have the same access key of the model. I use cmlapi. Can I do cmlapi.CreateModelBuildRequest with exiting model id? Do you understand what I mean?
... View more
05-23-2023
05:37 AM
@suhaa Just sent you a private message to discuss
... View more
05-23-2023
05:28 AM
Happy to see you got the issue resolved. Thanks for marking the solution and best of luck moving forward. 😊
... View more
05-22-2023
09:07 AM
What you are doing is mostly correct. I see a couple of items that may be creating some issues, e.g. mis-assigned values. For example, you are assigning accel.x to all three attributes values ax, ay, az. But it seems like you probably meant to assign ax = $.accel.x , ay =$.accel.y , az =$.accel.z It seems you've mixed up "random" and "rand". The field name in your JSON is "random" so your attribute assignment should be rand = $.random Where is "btn" in your JSON?
... View more
05-22-2023
06:50 AM
1 Kudo
Congratulations on resolving the issue @cdsw_user_23432. Please use the "Accept as Solution" button to mark the reply that resolved the issue for you. It helps others in a similar situation find assistance in the future.
... View more