Member since
07-30-2019
3387
Posts
1617
Kudos Received
999
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 84 | 11-05-2025 11:01 AM | |
| 348 | 10-20-2025 06:29 AM | |
| 488 | 10-10-2025 08:03 AM | |
| 350 | 10-08-2025 10:52 AM | |
| 378 | 10-08-2025 10:36 AM |
02-22-2017
10:16 PM
Thanks, this is exactly what I'm looking for!
... View more
02-22-2017
01:36 PM
2 Kudos
@Pradhuman Gupta Which "Protocol" are you using in your PutSplunk processor.
There is no assurance or delivery if you are using UDP. With TCP protocol there is confirmed delivery. You could use NiFi's provenance to track the FlowFile's processed by the PutSplunk processor. This will allow you to get the details on FlowFiles that have "SEND" provenance events associated to them. Thanks, Matt
... View more
02-24-2017
10:32 PM
update: I finally figured out how to setup IBM MQ for Nifi Here is my steps to setup IBM MQ for the Nifi connection 1) Create queue manager myMQManager using MQ Explorer 2) create Channel and queue using the following commands runmqsc myMQManager def chl(myMQChannel) chltype(svrconn) replace def chl(myMQChannel) chltype(clntconn) conname('localhost(1420)') QMNAME('myMQManager') replace def ql(myMQqueue) 2) Grant necessary authorization for a user using MQ Explorer 2.1 Queue Manager Authorities Setting right click your queue manager-->Object Authorities-->Add Role Based Authorities -->give the user access permission 2.2 Queue Authorities Setting right click your queue-->Object Authorities-->Manage Authority Records-->Specific Profiles--> your queue--> Accumulated authorities-->setup your user Access Permission for the queue 3) After the configuration and setup your Nifi ConsumeJMS processor, you should be able connect to IBM MQ
... View more
05-05-2017
10:00 PM
@Pradhuman Gupta You cannot setup logging for a specific processor. But you can setup a new logger for a specific processor class. First you would create a new appender in the nifi logback.xml file: <appender name="PROCESSOR_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-processsor.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--
For daily rollover, use 'user_%d.log'.
For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'.
To GZIP rolled files, replace '.log' with '.log.gz'.
To ZIP rolled files, replace '.log' with '.log.zip'.
-->
<fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-processor_%d.log</fileNamePattern>
<!-- keep 5 log files worth of history -->
<maxHistory>5</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{120} %msg%n</pattern>
<immediateFlush>true</immediateFlush>
</encoder>
</appender>
Then you create a new logger that will write to that appender log file: <logger name="org.apache.nifi.processors.attributes.UpdateAttribute" level="WARN" additivity="false">
<appender-ref ref="PROCESSOR_FILE"/>
</logger> In the above example i am creating a logger for the UpdateAttribute processor. Now any WARN or ERROR log messages produced by this specific processor will be written to this new log. You can expand upon this flow by configuring loggers for each Processor class you want to monitor and send them to the same appender. Then use a SplitText processor to split the content of the FlowFile produced by the TailFile. then use Route On Content processor to route specific log lines produced by each processor class to a different put email or simply create a different message body attribute for each. Thanks, Matt
... View more
02-22-2017
01:54 PM
@mayki wogno No problem... If one of the answers help drive you to a solution to your question, please accept that answer to help drive this community forward.
... View more
02-28-2017
03:23 PM
Thanks @Matt Clarke
... View more
02-16-2017
05:11 PM
1 Kudo
@Anshuman Ghosh The Search Value Regex above has 4 capture groups from a valid IP address. Each capture group can then be referenced in the replacement Value as $1, $2, $3, and/or $4. In the example above the replacement for each found valid IP is still the first two numbers followed by ".x.x". You can of course change the replacement value to whatever meets your specific needs. Thanks, Matt
... View more
03-17-2017
03:53 PM
Can you use the "matches" function in expression language? It should allow any regular expression: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#matches
... View more
02-16-2017
03:44 PM
@marksf Correct, the "key" icon will allow you to apply component level access policies to your process group with id "863928d5-12e7-...." Once "view the component" is granted the ID will be replaced with the actual process group name.
"Modify the component" will allow the user to also add, modify and delete components within that process group. Thanks, Matt
... View more
02-16-2017
02:07 PM
Thanks for the config tip, really helpful. It eventually came up by the morning. I think the issue is that my Hadoop cluster is down and I have at least 50 connections to it. I imagine a few minute time out per connection, 3-6 connections each, that's a big chunk of time.
... View more