Member since
05-16-2019
25
Posts
1
Kudos Received
0
Solutions
11-08-2017
10:19 AM
Thanks Matt, I'll be running tests today, thank you very much for your answer.
... View more
11-07-2017
12:22 PM
Hello. I have seen that for there is a way to get more speed from the script that we use in an executescript processor, Matt Burgess comments that it is possible to use the InvokeScriptedProcessor processor to get it. In fact there is a template generated in groovy to be able to insert the imports and code of our script. ////////////////////////////////////////////////////////////
// imports go here
////////////////////////////////////////////////////////////
class E{ void executeScript(session, context, log, REL_SUCCESS, REL_FAILURE)
{
////////////////////////////////////////////////////////////
// your code goes here
////////////////////////////////////////////////////////////
}
}
class GroovyProcessor implements Processor {
def REL_SUCCESS = new Relationship.Builder().name("success").description('FlowFiles that were successfully processed are routed here').build()
def REL_FAILURE = new Relationship.Builder().name("failure").description('FlowFiles that were not successfully processed are routed here').build()
def ComponentLog log
def e = new E()
void initialize(ProcessorInitializationContext context) { log = context.logger }
Set<Relationship> getRelationships() { return [REL_FAILURE, REL_SUCCESS] as Set }
Collection<ValidationResult> validate(ValidationContext context) { null }
PropertyDescriptor getPropertyDescriptor(String name) { null }
void onPropertyModified(PropertyDescriptor descriptor, String oldValue, String newValue) { }
List<PropertyDescriptor> getPropertyDescriptors() { null }
String getIdentifier() { null }
void onTrigger(ProcessContext context, ProcessSessionFactory sessionFactory) throws ProcessException {
def session = sessionFactory.createSession()
try {
e.executeScript(session, context, log, REL_SUCCESS, REL_FAILURE)
session.commit()
} catch (final Throwable t) {
log.error('{} failed to process due to {}; rolling back session', [this, t] as Object[])
session.rollback(true)
throw t
}}}
processor = new GroovyProcessor()
Unfortunately I haven't been able to find the same template but for jython. You know where the template is located or if possible someone could generate a template that can be used for jython code. Thank you very much. I enclose the link to Matt Burgess' article in case anyone would like to go into more detail. Link
... View more
Labels:
- Labels:
-
Apache NiFi
10-16-2017
01:52 PM
Hi Abdelkrim Hadjidj, thanks for the help. Unfortunately, it's not a solution to the problem I have. Since as a target I want to reduce the number of processors to use. Having many kafta topics, I think the solution may be to use the executescript processor. Since if I generate attributes in another way, I will end up having only one publisher, but many updateattribute. Greetings
... View more
10-16-2017
01:04 PM
Hi in Apache Nifi using the processor' executescript' I want to open the content of the flowfile, and according to if it contains any specific word. Generate an attribute, which I can use with another processor. Example of flowfile content: ford, mustang, 67,350, texas, blue, good, etc. Objective: If the line contains the term ford. I want to generate an attribute with the name kafka_topic with the value' Ford'. If the line contains the term audi. I want to generate an attribute with the name kafka_topic with the value' Audi'. That way in the next processor:
I will be able to use the value of ${kafka_topic} in the next processor and receive the correct value as the condition is met. I know how to read the content of the flowfile, but I've been blocked in the part where I have to generate the attribute. someone please can help me. I actually have this, for what it's worth: from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import StreamCallback
class PyStreamCallback(StreamCallback):
def __init__(self):
pass
def process(self, inputStream, outputStream):
Log = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
Log2 = str(Log).split(',')
Brand = Log2[0]
Color = Log2[5]
Model = Log2[1]
if Brand == 'ford' and Color == 'gray':
NewLog = str(Log2)
outputStream.write(bytearray((NewLog).encode('utf-8')))
if Brand == 'audi' and Color == 'black':
NewLog = str(Log2)
outputStream.write(bytearray((NewLog).encode('utf-8')))
if Brand == 'bmw' and Color == 'white':
NewLog = str(Log2)
outputStream.write(bytearray((NewLog).encode('utf-8')))
... View more
Labels:
- Labels:
-
Apache NiFi
10-16-2017
10:16 AM
At the moment with my little knowledge of nifi, I'm trying to generate an attribute from the executescript processor. At the moment I am able to read the content of the flowfile and have generated conditions using' if' and' else'. Unfortunately, at the moment I am not able to generate the dynamic attribute. so that the flowfile that comes out, comes out as it is, but with a dynamic attribute that allows pusblishkafka to derive the content.
... View more
10-16-2017
10:10 AM
Hello shu, as usual your answer is very clear and well worked.
In fact what you say is effective, but unfortunately it would not be practical for my particular case. While it is true that using this method you describe, only one publkafka processor is required. It will generate as many updateattribute processors as there are kafka topics. Please excuse me if my way of expressing myself in the question was not the most appropriate. finally the goal I'm looking for is not to have dozens of processors.
Thank you for your help, I always learn something from your answers.
... View more
10-16-2017
08:35 AM
Hello, everybody.
I hope you guys will please help me.
I have to store data in many different kafka topics in order to not have dozens of publishkafta processors. I thought I could use a variable in the topic name property.
My question is as follows, if the log lines I receive have to meet various conditions to be assigned to a topic or other topic. How it could generate the attribute that will serve as a variable to the publkafka processor. Example: - adult,dog,bulldog,9,23,male,brown,4,etc
If my log line contains two terms, let's say 'cat' and 'adult' will have to generate the variable 'CAT1' for publishkafka. If my log line contains the term 'adult' or 'puppy' and also the term 'dog' will have to generate the variable' DOGALL' for publkafka. I understand that I could use the executescript processor. Someone can help me with the problem, please. Greetings
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache NiFi
10-15-2017
07:45 PM
That's a great answer. Thank you so much for the help. It'll really come in handy. I can see that he has taken time to respond, and I thank him for that.
... View more
10-15-2017
01:58 PM
Hello, I have a set of syslog lines coming in one by one. Since I want to send them to different processors for storage, I want to derive them according to a series of conditions. A fellow member of the community indicated to me that he could use the RouteOnContent processor. In fact I have derived the logs according to a contained word towards one site or another. Property Country = (. *USA. *) But the problem I have now is that I find it necessary that the condition to derive is more than one word and I am not sure how to do it. e. g. my log line is as follows: 2017, michigan, january, rainy, 20, eeuu, cloudy I want to send to the site' A' if it contains the word January and USA. I want to send to site' B' if it contains January and cloudy I want to send to site' C' if it contains January or February and the USA. Thank you so much for the help.
... View more
Labels:
- Labels:
-
Apache NiFi
10-13-2017
12:45 PM
It worked perfectly. Thanks!
... View more
- « Previous
-
- 1
- 2
- Next »