Member since
02-07-2019
2755
Posts
241
Kudos Received
31
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2972 | 08-21-2025 10:43 PM | |
| 3113 | 04-15-2025 10:34 PM | |
| 8344 | 10-28-2024 12:37 AM | |
| 2794 | 09-04-2024 07:38 AM | |
| 4864 | 06-10-2024 10:24 PM |
06-22-2023
02:11 AM
HI @damon The most likely culprit, in this case, would be the Packet length crossing the jute buffer limit. As this is seen for the Kms service, verify if you are getting any warnings such as "Packet len is out of range!" If you do, then raise the jute buffer for the Zookeeper service as well as for the Kms service ( client ) and restart the respective services to resolve this issue.
... View more
06-21-2023
01:04 AM
1 Kudo
@diephan97, Welcome to our community! To help you get the best possible answer, I have tagged in our CDP experts @Gopinath @smdas who may be able to assist you further. Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.
... View more
06-20-2023
02:30 AM
1 Kudo
The problem is solved. Follow the below steps to fix it. sudo nano /etc/ssh/sshd_config In the file now look for PermitRootLogin without-password and replace it with this PermitRootLogin yes Now restart the ssh service sudo service ssh restart retry the installation and it should work.
... View more
06-19-2023
11:44 PM
@eykf, I never tried to send messages from NiFi to MS Teams but upon reading the documentation, I am not quite sure you need a Webhook URL at all. MS Teams provides the Graph API to be able to integrate MS Teams into external application: https://learn.microsoft.com/en-us/graph/api/resources/teams-api-overview?view=graph-rest-1.0 Now, assuming that you have the channel and the users already created, my first guess is that you would need only to send the message so basically all you have to do is create the API POST as described in the following link: https://learn.microsoft.com/en-us/graph/api/chatmessage-post?view=graph-rest-1.0&tabs=http And here are the parameters in case you need a more complex message, than the one described in the above link: https://learn.microsoft.com/en-us/graph/api/resources/chatmessage?view=graph-rest-1.0
... View more
06-18-2023
10:40 PM
Thanks for your response @habibalsa You may considering getting a custom runtime add-on created with the packages. Still you need to work with CML admin to have them added in to CML. https://docs.cloudera.com/machine-learning/cloud/runtimes/topics/ml-custom-runtime-addons.html
... View more
06-16-2023
07:23 AM
@bhadraka What version of NiFi are you using? In NiFi 1.20.0, you can use ReplaceText Processor after reading in the file. Using the line-by-line evaluation mode, there is a drop down "Except-Last-Line". You could then configure it to just replace all previous lines with empty strings. Here's a screenshot of my ReplaceText processor properties.
... View more
06-16-2023
06:25 AM
@MOUROU Is your NiFi configured to support Oauth2 based user authentication? It looks more like you are using either kerberos-provider or ldap-provider fro user authentication. My suggestion to create a client certificate and use a SSLContext service for client authentication for an automated dataflow like this is because: 1. No need to obtain an token ever. 2. Certs can be created with long expiration time. 3. Tokens are NiFi node specific (same token is not valid for a different NiFi node in a the same NiFi cluster). 4. Same certs works no matter which node in the cluster the invokeHTTP connects with. Matt
... View more
06-15-2023
11:41 AM
This simple InvokeScriptedProcessor will look for a FlowFile attribute called "ip_address" and will attempt the reverse lookup and create a new attribute called "host_name" with the resolved value. import groovy.json.JsonOutput
import groovy.json.JsonSlurper
import java.net.InetAddress
import java.net.UnknownHostException
import java.nio.charset.StandardCharsets
import org.apache.commons.io.IOUtils
class GroovyProcessor implements Processor {
PropertyDescriptor BATCH_SIZE = new PropertyDescriptor.Builder()
.name("BATCH_SIZE")
.displayName("Batch Size")
.description("The number of incoming FlowFiles to process in a single execution of this processor.")
.required(true)
.defaultValue("1000")
.addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR)
.build()
Relationship REL_SUCCESS = new Relationship.Builder()
.name("success")
.description('FlowFiles that were successfully processed are routed here')
.build()
Relationship REL_FAILURE = new Relationship.Builder()
.name("failure")
.description('FlowFiles that were not successfully processed are routed here')
.build()
ComponentLog log
void initialize(ProcessorInitializationContext context) { log = context.logger }
Set<Relationship> getRelationships() { return [REL_FAILURE, REL_SUCCESS] as Set }
Collection<ValidationResult> validate(ValidationContext context) { null }
PropertyDescriptor getPropertyDescriptor(String name) { null }
void onPropertyModified(PropertyDescriptor descriptor, String oldValue, String newValue) { }
List<PropertyDescriptor> getPropertyDescriptors() { Collections.unmodifiableList([BATCH_SIZE]) as List<PropertyDescriptor> }
String getIdentifier() { null }
JsonSlurper jsonSlurper = new JsonSlurper()
JsonOutput jsonOutput = new JsonOutput()
def reverseDnsLookup(String ipAddress) {
try {
InetAddress inetAddress = InetAddress.getByName(ipAddress)
String hostName = inetAddress.getCanonicalHostName()
return hostName
} catch (UnknownHostException e) {
return "Unknown"
}
}
void onTrigger(ProcessContext context, ProcessSessionFactory sessionFactory) throws ProcessException {
ProcessSession session = sessionFactory.createSession()
try {
List<FlowFile> flowFiles = session.get(context.getProperty(BATCH_SIZE).asInteger())
if (!flowFiles) return
Map customAttributes = [:]
flowFiles.each { flowFile ->
String ipAddress = flowFile.getAttribute("ip_address")
if (ipAddress) {
String hostName = reverseDnsLookup(ipAddress)
customAttributes["host_name"] = hostName
flowFile = session.putAllAttributes(flowFile, customAttributes)
}
session.transfer(flowFile, REL_SUCCESS)
}
session.commit()
} catch (final Throwable t) {
log.error('{} failed to process due to {}; rolling back session', [this, t] as Object[])
session.rollback(true)
throw t
}
}
}
processor = new GroovyProcessor()
... View more
06-14-2023
02:42 PM
Consume Kafka has an attribute called "Message Demarcator"...click on it hold shift+enter and instead of pulling 1 event at a time it'll create a single FlowFile with several events at a time and might make your merge even better. You can do the same thing with the Publish...merge on shift+enter and configure the same demarcator and you'll achieve greater throughput
... View more