Member since
05-26-2021
14
Posts
1
Kudos Received
0
Solutions
01-30-2023
05:43 AM
@myuintelli2021 Hello Ming, NiFi 1.15.3 will support JDK 1.8 or 1.11. We do strongly encourage users to be on the latest update version of either of those with NiFI. So not sure what update release of JDK your are on and which JDK provider (Oracle, OpenJDK, etc) you are using. I unfortunately do not have access ti a Windows 2019 datacenter edition to see if I can reproduce myself to evaluate further. I strongly encourage you to raise a community question to take your query further. A new question will gather more attention rather than trying to diagnosis and resolve your specific issue with the comments of a community article. Thank you, Matt
... View more
01-19-2023
07:29 AM
Hi! We have a stand alone nifi server v1.15.3 deployed to Windows server 2019 data center edition. It runs on java 1.8.0_33. We recently noticed that the nifi service does not seem to pick up the log rotation and retention settings in C:\nifi-1.15.3\conf\logback.xml. We have received some help under https://community.cloudera.com/t5/Community-Articles/Understanding-how-the-logback-xml-configuration-in-NiFi/tac-p/361419#M6616 and tested the configuration that master guru tested under nifi 1.19.1; unfortunately, the log rotation did not happen. Here is the logback.xml I am using: <?xml version="1.0" encoding="UTF-8"?> <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <configuration scan="true" scanPeriod="30 seconds"> <shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook" /> <contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator"> <resetJUL>true</resetJUL> </contextListener> <appender name="APP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app.log</file> <rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy"> <!-- For daily rollover, use 'app_%d.log'. For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'. To GZIP rolled files, replace '.log' with '.log.gz'. To ZIP rolled files, replace '.log' with '.log.zip'. --> <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern> <maxFileSize>2MB</maxFileSize> <!-- keep 30 log files worth of history --> <maxHistory>10</maxHistory> </rollingPolicy> <immediateFlush>true</immediateFlush> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> </encoder> </appender> <appender name="USER_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-user.log</file> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <!-- For daily rollover, use 'user_%d.log'. For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'. To GZIP rolled files, replace '.log' with '.log.gz'. To ZIP rolled files, replace '.log' with '.log.zip'. --> <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-user_%d.log</fileNamePattern> <!-- keep 30 log files worth of history --> <maxHistory>30</maxHistory> </rollingPolicy> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> </encoder> </appender> <appender name="BOOTSTRAP_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-bootstrap.log</file> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <!-- For daily rollover, use 'bootstrap_%d.log'. For hourly rollover, use 'bootstrap_%d{yyyy-MM-dd_HH}.log'. To GZIP rolled files, replace '.log' with '.log.gz'. To ZIP rolled files, replace '.log' with '.log.zip'. --> <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-bootstrap_%d.log</fileNamePattern> <!-- keep 5 log files worth of history --> <maxHistory>5</maxHistory> </rollingPolicy> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> </encoder> </appender> <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender"> <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"> <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> </encoder> </appender> <!-- valid logging levels: TRACE, DEBUG, INFO, WARN, ERROR --> <logger name="org.apache.nifi" level="INFO"/> <logger name="org.apache.nifi.processors" level="WARN"/> <logger name="org.apache.nifi.processors.standard.LogAttribute" level="INFO"/> <logger name="org.apache.nifi.processors.standard.LogMessage" level="INFO"/> <logger name="org.apache.nifi.controller.repository.StandardProcessSession" level="WARN" /> <logger name="org.apache.zookeeper.ClientCnxn" level="ERROR" /> <logger name="org.apache.zookeeper.server.NIOServerCnxn" level="ERROR" /> <logger name="org.apache.zookeeper.server.NIOServerCnxnFactory" level="ERROR" /> <logger name="org.apache.zookeeper.server.NettyServerCnxnFactory" level="ERROR" /> <logger name="org.apache.zookeeper.server.quorum" level="ERROR" /> <logger name="org.apache.zookeeper.ZooKeeper" level="ERROR" /> <logger name="org.apache.zookeeper.server.PrepRequestProcessor" level="ERROR" /> <logger name="org.apache.nifi.controller.reporting.LogComponentStatuses" level="ERROR" /> <logger name="org.apache.calcite.runtime.CalciteException" level="OFF" /> <logger name="org.apache.curator.framework.recipes.leader.LeaderSelector" level="OFF" /> <logger name="org.apache.curator.ConnectionState" level="OFF" /> <!-- Logger for managing logging statements for nifi clusters. --> <logger name="org.apache.nifi.cluster" level="INFO"/> <!-- Logger for logging HTTP requests received by the web server. --> <logger name="org.apache.nifi.server.JettyServer" level="INFO"/> <!-- Logger for managing logging statements for jetty --> <logger name="org.eclipse.jetty" level="INFO"/> <!-- Suppress non-error messages due to excessive logging by class or library --> <logger name="org.springframework" level="ERROR"/> <!-- Suppress non-error messages due to known warning about redundant path annotation (NIFI-574) --> <logger name="org.glassfish.jersey.internal.Errors" level="ERROR"/> <!-- Suppress non-error messages due to Jetty AnnotationParser emitting a large amount of WARNS. Issue described in NIFI-5479. --> <logger name="org.eclipse.jetty.annotations.AnnotationParser" level="ERROR"/> <!-- Suppress non-error messages from SSHJ which was emitting large amounts of INFO logs by default --> <logger name="net.schmizz.sshj" level="WARN" /> <logger name="com.hierynomus.sshj" level="WARN" /> <!-- Suppress non-error messages from SMBJ which was emitting large amounts of INFO logs by default --> <logger name="com.hierynomus.smbj" level="WARN" /> <!-- Suppress non-error messages from AWS KCL which was emitting large amounts of INFO logs by default --> <logger name="com.amazonaws.services.kinesis" level="WARN" /> <!-- Suppress non-error messages from Apache Atlas which was emitting large amounts of INFO logs by default --> <logger name="org.apache.atlas" level="WARN"/> <!-- These log messages would normally go to the USER_FILE log, but they belong in the APP_FILE --> <logger name="org.apache.nifi.web.security.requests" level="INFO" additivity="false"> <appender-ref ref="APP_FILE"/> </logger> <!-- Logger for capturing user events. We do not want to propagate these log events to the root logger. These messages are only sent to the user-log appender. --> <logger name="org.apache.nifi.web.security" level="INFO" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <logger name="org.apache.nifi.web.api.config" level="INFO" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <logger name="org.apache.nifi.authorization" level="INFO" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <logger name="org.apache.nifi.cluster.authorization" level="INFO" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <logger name="org.apache.nifi.web.filter.RequestLogger" level="INFO" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <logger name="org.apache.nifi.web.api.AccessResource" level="INFO" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <logger name="org.springframework.security.saml.log" level="WARN" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <logger name="org.opensaml" level="WARN" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <!-- Logger for capturing Bootstrap logs and NiFi's standard error and standard out. --> <logger name="org.apache.nifi.bootstrap" level="INFO" additivity="false"> <appender-ref ref="BOOTSTRAP_FILE" /> </logger> <logger name="org.apache.nifi.bootstrap.Command" level="INFO" additivity="false"> <appender-ref ref="CONSOLE" /> <appender-ref ref="BOOTSTRAP_FILE" /> </logger> <!-- Everything written to NiFi's Standard Out will be logged with the logger org.apache.nifi.StdOut at INFO level --> <logger name="org.apache.nifi.StdOut" level="INFO" additivity="false"> <appender-ref ref="BOOTSTRAP_FILE" /> </logger> <!-- Everything written to NiFi's Standard Error will be logged with the logger org.apache.nifi.StdErr at ERROR level --> <logger name="org.apache.nifi.StdErr" level="ERROR" additivity="false"> <appender-ref ref="BOOTSTRAP_FILE" /> </logger> <root level="INFO"> <appender-ref ref="APP_FILE" /> </root> </configuration> Appreciate it if anyone can help or share expertise.
... View more
Labels:
- Labels:
-
Apache NiFi
03-28-2022
08:35 AM
By checking the status of https://issues.apache.org/jira/browse/NIFIREG-313, it seems that the feature of authenticating using OIDC for nifi registry is already resolved as of 9/14/2020. But In nifi registry admin guide at https://nifi.apache.org/docs/nifi-registry-docs/index.html, it still mentions that only LDAP and Kerberos are supported. Is the document up-to-date?
... View more
02-28-2022
03:45 PM
@myuintelli2021 , Good point! I hadn't noticed that. Alternatively, you can download the NiFi Toolkit and use the encrypt-config.bat to perform the key migration. Check this out: https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#sensitive-property-key-migration The toolkit contains a Windows version of the script (encrypt-config.bat). HTH, André
... View more
06-14-2021
11:56 AM
1 Kudo
@VidyaSargur We have not resolved the issue yet due to some priority shift. We're still seeing the same error. Will follow up on it soon. Ming
... View more
06-07-2021
07:50 AM
@myuintelli2021 Assisting on your new post here: https://community.cloudera.com/t5/Support-Questions/Nifi-untrusted-proxy-caused-by-Untrusted-Proxy-Exception/m-p/317796/highlight/false#M227327 Your choice of user authentication does not matter here. Authentication and Authorization processes are handled independently of one another. The Authentication of users/clients results in a string which is evaluated against identity mapping properties and then passed to the configured authorizer for authorization. Your exception points and missing /proxy authorization for your node strings. Hope this helps, Matt
... View more