Member since
03-05-2021
19
Posts
3
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 4996 | 07-22-2022 08:59 AM |
01-10-2023
06:10 PM
1 Kudo
Hi @edaley, I have a couple instances runnning on Windows and all of them rotates uses the TimeBasedRollingPolicy with the following configuration <appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app.log</file -->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app_%d.log.zip</fileNamePattern>
<maxHistory>30</maxHistory>
</rollingPolicy>
<immediateFlush>true</immediateFlush>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%date %level [%thread] %logger{40} %msg%n</pattern>
</encoder> This configuration works over the nifi-app.log file and move it into a new file nifi-app_date.log.zip at midnight. Hope this helps.
... View more
07-22-2022
08:59 AM
Hi All, The problem was on the JDK version. We were using OpenJDK 11.0.2 which had a bug in the TLS handshake. Solution: Upgrade JDK (now using 11.0.15).
... View more
07-12-2022
06:44 PM
Does the target URL hostname go to a LB that can route that request to any number of endpoint servers? I don't really know, but thing that this is the scenario. Perhaps not all those endpoint servers use the same serverAuth certificate or are not all signed by the same authority or an authority known the the truststore configured in the SSLContextService you have configured in your invokeHTTP processor. Is possible. I added the root certificate available on CA Authority to the keystore and will monitor the behavior.
... View more
07-12-2022
05:44 PM
Hi @araujo, Just today I detected more than 150 errors over around 400 HTTP request.
... View more
07-06-2022
12:38 PM
Hi, I am very frequently getting the following exception whenever I call the URL seen in the message. 2022-07-06 15:24:56,138 ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.InvokeHTTP InvokeHTTP[id=33351c02-15ee-1ccc-6c00-dda0e16110ca] Routing to Failure due to exception: javax.net.ssl.SSLPeerUnverifiedException: Hostname creatorapp.zoho.com not verified (no certificates): javax.net.ssl.SSLPeerUnverifiedException: Hostname creatorapp.zoho.com not verified (no certificates)
javax.net.ssl.SSLPeerUnverifiedException: Hostname creatorapp.zoho.com not verified (no certificates)
at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.kt:396)
at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.kt:337)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:209)
at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:226)
at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:106)
at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:74)
at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:255)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:32)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109)
at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201)
at okhttp3.internal.connection.RealCall.execute(RealCall.kt:154)
at org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:850)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1202)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834) Some details about it: 1. The URL certificate is valid 2. The request is processed correctly on the second or third try and if several flowfiles are queued, the following messages may or may not present the same behavior. But usually the request is always processed. 3. I am using the self-signed certificate generated by Nifi. Thanks.
... View more
Labels:
- Labels:
-
Apache NiFi
04-02-2022
02:14 PM
Hi André, It works, however I'm still facing problems to start the MiNiFi, now I'm seeing the following errors in the nifi-app.log 2022-04-02 17:25:56,124 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 22, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,140 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 75, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,140 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 126, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,140 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 177, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,155 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 212, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,155 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 242, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,155 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 277, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,155 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 359, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,155 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 407, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,155 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 446, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,171 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 486, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,171 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 549, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,171 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 612, col 29: cvc-complex-type.2.4.a: Invalid content was found starting with element 'maxConcurrentTasks'. One of '{bundle}' is expected.
2022-04-02 17:25:56,191 WARN [main] org.apache.nifi.util.FlowParser Schema validation error parsing Flow Configuration at line 680, col 28: cvc-complex-type.2.4.a: Invalid content was found starting with element 'networkInterface'. One of '{inputPort, outputPort}' is expected.
2022-04-02 17:25:56,311 INFO [main] o.a.nifi.headless.HeadlessNiFiServer Loading Flow...
2022-04-02 17:25:56,343 WARN [main] o.a.nifi.headless.HeadlessNiFiServer Failed to start headless server... shutting down.
java.lang.Exception: Unable to load flow due to: java.lang.IllegalArgumentException: NiFi Sensitive Properties Key [nifi.sensitive.props.key] is required
at org.apache.nifi.headless.HeadlessNiFiServer.start(HeadlessNiFiServer.java:166)
at org.apache.nifi.minifi.MiNiFi.<init>(MiNiFi.java:163)
at org.apache.nifi.minifi.MiNiFi.<init>(MiNiFi.java:64)
at org.apache.nifi.minifi.MiNiFi.main(MiNiFi.java:265)
Caused by: java.lang.IllegalArgumentException: NiFi Sensitive Properties Key [nifi.sensitive.props.key] is required
at org.apache.nifi.encrypt.PropertyEncryptorFactory.getPropertyEncryptor(PropertyEncryptorFactory.java:42)
at org.apache.nifi.headless.HeadlessNiFiServer.start(HeadlessNiFiServer.java:125)
... 3 common frames omitted
2022-04-02 17:25:56,343 INFO [Thread-1] org.apache.nifi.minifi.MiNiFi Initiating shutdown of MiNiFi server...
2022-04-02 17:25:56,343 WARN [Thread-1] o.a.nifi.headless.HeadlessNiFiServer Problem occurred ensuring flow controller or repository was properly terminated due to java.lang.NullPointerException
2022-04-02 17:25:56,343 ERROR [Thread-1] org.apache.nifi.BootstrapListener Failed to communicate with Bootstrap. Bootstrap may be unable to issue or receive commands from MiNiFi
2022-04-02 17:25:56,343 INFO [Thread-1] org.apache.nifi.minifi.MiNiFi MiNiFi server shutdown completed (nicely or otherwise). About the warnings I could not find which processors in the flow has the maxconcurrentask unset, all have one as value (1). About the network interface, input and output ports warnings I suppose that it refers to the remote process group. In this point, I understand that, at least the network interface, is an optional parameter. The final error, nifi.sensitive.props.key. I seted this property but when minifi try to starts set the value to empty again and then fails. Thanks again.
... View more
04-01-2022
07:05 PM
Hi, I'm trying to deploy a minifi instance in Windows Server. The process deployed basically waits for a file, read it and send the content to the Nifi instance in other server in the same network. Template's transformation finished OK and none changes have been made D:\minifi-1.14.0\minifi-toolkit-1.14.0\bin>config.bat transform Captura_Log_de_Errrores_Remotos.xml Captura_Log_de_Errrores_Remotos.yml
No validation errors found in converted configuration. My problem is that MiNiFi is not starting with the following exception: c:\minifi\bin>run-minifi.bat
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/minifi/lib/bootstrap/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/minifi/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
Exception in thread "main" java.io.IOException: Unable to successfully transform the provided configuration
at org.apache.nifi.minifi.bootstrap.RunMiNiFi.performTransformation(RunMiNiFi.java:1703)
at org.apache.nifi.minifi.bootstrap.RunMiNiFi.start(RunMiNiFi.java:1180)
at org.apache.nifi.minifi.bootstrap.RunMiNiFi.main(RunMiNiFi.java:244)
Caused by: java.io.IOException: org.yaml.snakeyaml.error.YAMLException: java.nio.charset.MalformedInputException: Input length = 1
at org.apache.nifi.minifi.commons.schema.serialization.SchemaLoader.loadYamlAsMap(SchemaLoader.java:64)
at org.apache.nifi.minifi.commons.schema.serialization.SchemaLoader.loadConvertableSchemaFromYaml(SchemaLoader.java:79)
at org.apache.nifi.minifi.bootstrap.util.ConfigTransformer.transformConfigFile(ConfigTransformer.java:94)
at org.apache.nifi.minifi.bootstrap.RunMiNiFi.performTransformation(RunMiNiFi.java:1693)
... 2 more
Caused by: org.yaml.snakeyaml.error.YAMLException: java.nio.charset.MalformedInputException: Input length = 1
at org.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:200)
at org.yaml.snakeyaml.reader.StreamReader.peek(StreamReader.java:146)
at org.yaml.snakeyaml.scanner.ScannerImpl.scanPlain(ScannerImpl.java:2000)
at org.yaml.snakeyaml.scanner.ScannerImpl.fetchPlain(ScannerImpl.java:1046)
at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:401)
at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:226)
at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:557)
at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:157)
at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:147)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:224)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:155)
at org.yaml.snakeyaml.composer.Composer.composeSequenceNode(Composer.java:199)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:153)
at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:246)
at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:237)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:225)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:155)
at org.yaml.snakeyaml.composer.Composer.composeSequenceNode(Composer.java:199)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:153)
at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:246)
at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:237)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:225)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:155)
at org.yaml.snakeyaml.composer.Composer.composeDocument(Composer.java:122)
at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:105)
at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:120)
at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:450)
at org.yaml.snakeyaml.Yaml.load(Yaml.java:381)
at org.apache.nifi.minifi.commons.schema.serialization.SchemaLoader.loadYamlAsMap(SchemaLoader.java:55)
... 5 more
Caused by: java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at org.yaml.snakeyaml.reader.UnicodeReader.read(UnicodeReader.java:123)
at java.io.Reader.read(Reader.java:140)
at org.yaml.snakeyaml.reader.StreamReader.update(StreamReader.java:184)
... 33 more Nifi's configuration is: Port 10000 is up on Nifi's server and can be accessed from MiniFi's server... I'll appreciate your help
... View more
Labels:
- Labels:
-
Apache NiFi
03-31-2022
10:06 AM
1 Kudo
Hi André, That is what I needed. Thanks a lot. For those who are facing the same problem in windows; @araujo's solution also works with Powershell
... View more
03-29-2022
12:34 PM
Hi, An external process creates an error.file on certain events and updates the process log file. The process that I'm creating starts when the error.file is created and need to read the last 10 - 15 lines of the log.file. My problem is that the log.file does not rollover and just grows continually and neither follows a log standard in order to use SysLog processors. How can I get that last 10 - 15 lines of the log file? Something like ( "tail -15 log.file" ). Thanks in advance.
... View more
Labels:
- Labels:
-
Apache NiFi