additivity="false" is essential Complete answer on StackOverflow. ... View more

Usually (as is the case for the nifi-hadoop-bundle), the NAR depends on the nifi-hadoop-libraries-nar, as it provides the Hadoop libraries (such as the provided dependencies you have in your processor POM like hadoop-common), and its parent NAR is nifi-standard-services-api-nar (which you have in your NAR POM). Currently, NARs can only have one parent, so you wouldn't be able to depend on both the hadoop-libraries and standard-services-api NARs at the same time. Since the former depends on the latter, this works for your processor. Try replacing the NAR POM dependency on nifi-standard-services-api-nar to nifi-hadoop-libraries-nar, this should provide all the classes/JARs/dependencies you need. ... View more

I got the answer at StackOverflow. I am able to create a logfile on local, Windows machine but the same config. is not working on the Linux env., maybe I have missed something. Below is the logback.xml used on my local machine : <?xml version="1.0" encoding="UTF-8"?> <configuration scan="true" scanPeriod="30 seconds"> <contextListener> <resetJUL>true</resetJUL> </contextListener> <appender name="APP_FILE"> <file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app.log</file> <rollingPolicy> <!-- For daily rollover, use 'app_%d.log'. For hourly rollover, use 'app_%d{yyyy-MM-dd_HH}.log'. To GZIP rolled files, replace '.log' with '.log.gz'. To ZIP rolled files, replace '.log' with '.log.zip'. --> <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-app_%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern> <maxFileSize>100MB</maxFileSize> <!-- keep 30 log files worth of history --> <maxHistory>30</maxHistory> </rollingPolicy> <immediateFlush>true</immediateFlush> <encoder> <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> </encoder> </appender> <appender name="USER_FILE"> <file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-user.log</file> <rollingPolicy> <!-- For daily rollover, use 'user_%d.log'. For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'. To GZIP rolled files, replace '.log' with '.log.gz'. To ZIP rolled files, replace '.log' with '.log.zip'. --> <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-user_%d.log</fileNamePattern> <!-- keep 30 log files worth of history --> <maxHistory>30</maxHistory> </rollingPolicy> <encoder> <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> </encoder> </appender> <appender name="BOOTSTRAP_FILE"> <file>${org.apache.nifi.bootstrap.config.log.dir}/nifi-bootstrap.log</file> <rollingPolicy> <!-- For daily rollover, use 'user_%d.log'. For hourly rollover, use 'user_%d{yyyy-MM-dd_HH}.log'. To GZIP rolled files, replace '.log' with '.log.gz'. To ZIP rolled files, replace '.log' with '.log.zip'. --> <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/nifi-bootstrap_%d.log</fileNamePattern> <!-- keep 5 log files worth of history --> <maxHistory>5</maxHistory> </rollingPolicy> <encoder> <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> </encoder> </appender> <appender name="CONSOLE"> <encoder> <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> </encoder> </appender> <!-- Start : Added for log for custom processor --> <appender name="SQLSERVER-CDC"> <file>${org.apache.nifi.bootstrap.config.log.dir}/sqlserver-cdc.log</file> <rollingPolicy> <fileNamePattern>${org.apache.nifi.bootstrap.config.log.dir}/sqlserver-cdc_%d.log</fileNamePattern> <maxHistory>30</maxHistory> </rollingPolicy> <encoder> <pattern>%date %level [%thread] %logger{40} %msg%n</pattern> </encoder> </appender> <!-- End : Added for log for custom processor --> <!-- valid logging levels: TRACE, DEBUG, INFO, WARN, ERROR --> <logger name="org.apache.nifi" level="INFO"/> <logger name="org.apache.nifi.processors" level="WARN"/> <logger name="org.apache.nifi.processors.standard.LogAttribute" level="INFO"/> <logger name="org.apache.nifi.controller.repository.StandardProcessSession" level="WARN" /> <logger name="org.apache.zookeeper.ClientCnxn" level="ERROR" /> <logger name="org.apache.zookeeper.server.NIOServerCnxn" level="ERROR" /> <logger name="org.apache.zookeeper.server.NIOServerCnxnFactory" level="ERROR" /> <logger name="org.apache.zookeeper.server.quorum" level="ERROR" /> <logger name="org.apache.zookeeper.ZooKeeper" level="ERROR" /> <logger name="org.apache.zookeeper.server.PrepRequestProcessor" level="ERROR" /> <logger name="org.apache.calcite.runtime.CalciteException" level="OFF" /> <logger name="org.apache.curator.framework.recipes.leader.LeaderSelector" level="OFF" /> <logger name="org.apache.curator.ConnectionState" level="OFF" /> <!-- Logger for managing logging statements for nifi clusters. --> <logger name="org.apache.nifi.cluster" level="INFO"/> <!-- Logger for logging HTTP requests received by the web server. --> <logger name="org.apache.nifi.server.JettyServer" level="INFO"/> <!-- Logger for managing logging statements for jetty --> <logger name="org.eclipse.jetty" level="INFO"/> <!-- Suppress non-error messages due to excessive logging by class or library --> <logger name="com.sun.jersey.spi.container.servlet.WebComponent" level="ERROR"/> <logger name="com.sun.jersey.spi.spring" level="ERROR"/> <logger name="org.springframework" level="ERROR"/> <!-- Suppress non-error messages due to known warning about redundant path annotation (NIFI-574) --> <logger name="com.sun.jersey.spi.inject.Errors" level="ERROR"/> <!-- Logger for capturing user events. We do not want to propagate these log events to the root logger. These messages are only sent to the user-log appender. --> <logger name="org.apache.nifi.web.security" level="INFO" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <logger name="org.apache.nifi.web.api.config" level="INFO" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <logger name="org.apache.nifi.authorization" level="INFO" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <logger name="org.apache.nifi.cluster.authorization" level="INFO" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <logger name="org.apache.nifi.web.filter.RequestLogger" level="INFO" additivity="false"> <appender-ref ref="USER_FILE"/> </logger> <!-- Logger for capturing Bootstrap logs and NiFi's standard error and standard out. --> <logger name="org.apache.nifi.bootstrap" level="INFO" additivity="false"> <appender-ref ref="BOOTSTRAP_FILE" /> </logger> <logger name="org.apache.nifi.bootstrap.Command" level="INFO" additivity="false"> <appender-ref ref="CONSOLE" /> <appender-ref ref="BOOTSTRAP_FILE" /> </logger> <!-- Everything written to NiFi's Standard Out will be logged with the logger org.apache.nifi.StdOut at INFO level --> <logger name="org.apache.nifi.StdOut" level="INFO" additivity="false"> <appender-ref ref="BOOTSTRAP_FILE" /> </logger> <!-- Everything written to NiFi's Standard Error will be logged with the logger org.apache.nifi.StdErr at ERROR level --> <logger name="org.apache.nifi.StdErr" level="ERROR" additivity="false"> <appender-ref ref="BOOTSTRAP_FILE" /> </logger> <!-- Start : Added for log for custom processor --> <logger name="com.datalake.processors.SQLServerCDCProcessor" level="DEBUG" > <appender-ref ref="SQLSERVER-CDC"/> </logger> <!-- End : Added for log for custom processor --> <root level="info"> <appender-ref ref="APP_FILE"/> </root> </configuration> ... View more