Support Questions

Find answers, ask questions, and share your expertise

setting nifi.python.command=python3 results in Failed to instantiate [org.apache.nifi.controller.FlowController]: Factory method 'flowController' threw exception with message: null

avatar
New Contributor

 

nifi.python.command=python3
nifi.flow.configuration.file=../data/flow.json.gz
nifi.flow.configuration.archive.enabled=false
nifi.flow.configuration.archive.dir=../data/archive/
nifi.flow.configuration.archive.max.time=30 days
nifi.flow.configuration.archive.max.storage=500 MB
nifi.flow.configuration.archive.max.count=
nifi.flowcontroller.autoResumeState=true
nifi.flowcontroller.graceful.shutdown.period=10 sec
nifi.flowservice.writedelay.interval=500 ms
nifi.administrative.yield.duration=30 sec
nifi.bored.yield.duration=10 millis
nifi.queue.backpressure.count=10000
nifi.queue.backpressure.size=1 GB
nifi.authorizer.configuration.file=./conf/authorizers.xml
nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
nifi.templates.directory=../data/templates
nifi.ui.banner.text=
nifi.ui.autorefresh.interval=30 sec
nifi.nar.library.directory=./lib
nifi.nar.library.autoload.directory=./extensions
nifi.nar.working.directory=./work/nar/
nifi.documentation.working.directory=./work/docs/components
nifi.nar.unpack.uber.jar=false
nifi.state.management.configuration.file=./conf/state-management.xml
nifi.state.management.provider.local=local-provider
nifi.state.management.provider.cluster=zk-provider
nifi.state.management.embedded.zookeeper.start=false
nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties
nifi.database.directory=./database_repository
nifi.h2.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE
nifi.repository.encryption.protocol.version=
nifi.repository.encryption.key.id=
nifi.repository.encryption.key.provider=
nifi.repository.encryption.key.provider.keystore.location=
nifi.repository.encryption.key.provider.keystore.password=
nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
nifi.flowfile.repository.wal.implementation=org.apache.nifi.wali.SequentialAccessWriteAheadLog
nifi.flowfile.repository.directory=./flowfile_repository
nifi.flowfile.repository.checkpoint.interval=20 secs
nifi.flowfile.repository.always.sync=false
nifi.flowfile.repository.retain.orphaned.flowfiles=true
nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
nifi.queue.swap.threshold=20000
nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository
nifi.content.claim.max.appendable.size=50 KB
nifi.content.repository.directory.default=./content_repository
nifi.content.repository.archive.max.retention.period=1 days
nifi.content.repository.archive.max.usage.percentage=60%
nifi.content.repository.archive.enabled=false
nifi.content.repository.always.sync=false
nifi.content.viewer.url=../nifi-content-viewer/
nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository
nifi.provenance.repository.directory.default=./provenance_repository
nifi.provenance.repository.max.storage.time=1 day
nifi.provenance.repository.max.storage.size=1 GB
nifi.provenance.repository.rollover.time=10 mins
nifi.provenance.repository.rollover.size=100 MB
nifi.provenance.repository.query.threads=2
nifi.provenance.repository.index.threads=2
nifi.provenance.repository.compress.on.rollover=true
nifi.provenance.repository.always.sync=false
nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship
nifi.provenance.repository.indexed.attributes=
nifi.provenance.repository.index.shard.size=500 MB
nifi.provenance.repository.max.attribute.length=65536
nifi.provenance.repository.concurrent.merge.threads=2
nifi.provenance.repository.buffer.size=100000
nifi.components.status.repository.implementation=org.apache.nifi.controller.status.history.VolatileComponentStatusRepository
nifi.components.status.repository.buffer.size=1440
nifi.components.status.snapshot.frequency=1 min
nifi.status.repository.questdb.persist.node.days=14
nifi.status.repository.questdb.persist.component.days=3
nifi.status.repository.questdb.persist.location=./status_repository
nifi.remote.input.host=MASKED
nifi.remote.input.secure=true
nifi.remote.input.socket.port=10443
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs
nifi.web.http.host=
nifi.web.http.network.interface.default=
nifi.web.https.host=MASKED
nifi.web.https.port=9443
nifi.web.https.network.interface.default=
nifi.web.https.application.protocols=http/1.1
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
nifi.web.max.header.size=16 KB
nifi.web.proxy.context.path=
nifi.web.proxy.host=MASKED
nifi.web.max.content.size=
nifi.web.max.requests.per.second=30000
nifi.web.max.access.token.requests.per.second=25
nifi.web.request.timeout=60 secs
nifi.web.request.ip.whitelist=
nifi.web.should.send.server.version=true
nifi.web.request.log.format=%{client}a - %u %t "%r" %s %O "%{Referer}i" "%{User-Agent}i"
nifi.web.https.ciphersuites.include=
nifi.web.https.ciphersuites.exclude=
nifi.sensitive.props.key=MASKED
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=NIFI_PBKDF2_AES_GCM_256
nifi.sensitive.props.provider=BC
nifi.sensitive.props.additional.keys=
nifi.security.keystore=/opt/nifi/data/certs/keystore.jks
nifi.security.keystoreType=jks
nifi.security.keystorePasswd=notmyactualpassword
nifi.security.keyPasswd=notmyactualpassword
nifi.security.truststore=/opt/nifi/data/certs/truststore.jks
nifi.security.truststoreType=jks
nifi.security.truststorePasswd=notmyactualpassword
nifi.security.user.authorizer=managed-authorizer
nifi.security.allow.anonymous.authentication=false
nifi.security.user.login.identity.provider=
nifi.security.user.jws.key.rotation.period=PT1H
nifi.security.ocsp.responder.url=
nifi.security.ocsp.responder.certificate=
nifi.security.user.oidc.discovery.url=
nifi.security.user.oidc.connect.timeout=5 secs
nifi.security.user.oidc.read.timeout=5 secs
nifi.security.user.oidc.client.id=
nifi.security.user.oidc.client.secret=
nifi.security.user.oidc.preferred.jwsalgorithm=
nifi.security.user.oidc.additional.scopes=
nifi.security.user.oidc.claim.identifying.user=
nifi.security.user.oidc.fallback.claims.identifying.user=
nifi.security.user.oidc.truststore.strategy=JDK
nifi.security.user.knox.url=
nifi.security.user.knox.publicKey=
nifi.security.user.knox.cookieName=hadoop-jwt
nifi.security.user.knox.audiences=
nifi.security.user.saml.idp.metadata.url=https://fs.lgt.com/FederationMetadata/2007-06/FederationMetadata.xml
nifi.security.user.saml.sp.entity.id=masked
nifi.security.user.saml.identity.attribute.name=Name ID
nifi.security.user.saml.group.attribute.name=http://schemas.xmlsoap.org/claims/Group
nifi.security.user.saml.request.signing.enabled=false
nifi.security.user.saml.want.assertions.signed=true
nifi.security.user.saml.signature.algorithm=http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
nifi.security.user.saml.authentication.expiration=12 hours
nifi.security.user.saml.single.logout.enabled=false
nifi.security.user.saml.http.client.truststore.strategy=JDK
nifi.security.user.saml.http.client.connect.timeout=30 secs
nifi.security.user.saml.http.client.read.timeout=30 secs
nifi.security.user.saml.message.logging.enabled=true
nifi.listener.bootstrap.port=0
nifi.cluster.protocol.heartbeat.interval=12 sec
nifi.cluster.protocol.heartbeat.missable.max=8
nifi.cluster.protocol.is.secure=true
nifi.cluster.is.node=true
nifi.cluster.node.address=MASKED
nifi.cluster.node.protocol.port=6007
nifi.cluster.node.protocol.max.threads=50
nifi.cluster.node.event.history.size=25
nifi.cluster.node.connection.timeout=30 sec
nifi.cluster.node.read.timeout=60 sec
nifi.cluster.node.max.concurrent.requests=100
nifi.cluster.firewall.file=
nifi.cluster.flow.election.max.wait.time=30 sec
nifi.cluster.flow.election.max.candidates=
nifi.cluster.load.balance.host=
nifi.cluster.load.balance.port=6342
nifi.cluster.load.balance.connections.per.node=1
nifi.cluster.load.balance.max.thread.count=8​
# Python 
nifi.python.command=python3
# Core Properties #
nifi.flow.configuration.file=../data/flow.json.gz
nifi.flow.configuration.archive.enabled=false
nifi.flow.configuration.archive.dir=../data/archive/
nifi.flow.configuration.archive.max.time=30 days
nifi.flow.configuration.archive.max.storage=500 MB
nifi.flow.configuration.archive.max.count=
nifi.flowcontroller.autoResumeState=true
nifi.flowcontroller.graceful.shutdown.period=10 sec
nifi.flowservice.writedelay.interval=500 ms
nifi.administrative.yield.duration=30 sec
# If a component has no work to do (is "bored"), how long should we wait before checking again for work?
nifi.bored.yield.duration=10 millis
nifi.queue.backpressure.count=10000
nifi.queue.backpressure.size=1 GB

nifi.authorizer.configuration.file=./conf/authorizers.xml
nifi.login.identity.provider.configuration.file=./conf/login-identity-providers.xml
nifi.templates.directory=../data/templates
nifi.ui.banner.text=
nifi.ui.autorefresh.interval=30 sec
nifi.nar.library.directory=./lib
nifi.nar.library.autoload.directory=./extensions
nifi.nar.working.directory=./work/nar/
nifi.documentation.working.directory=./work/docs/components
nifi.nar.unpack.uber.jar=false

####################
# State Management #
####################
nifi.state.management.configuration.file=./conf/state-management.xml
# The ID of the local state provider
nifi.state.management.provider.local=local-provider
# The ID of the cluster-wide state provider. This will be ignored if NiFi is not clustered but must be populated if running in a cluster.
nifi.state.management.provider.cluster=zk-provider
# Specifies whether or not this instance of NiFi should run an embedded ZooKeeper server
nifi.state.management.embedded.zookeeper.start=false
# Properties file that provides the ZooKeeper properties to use if <nifi.state.management.embedded.zookeeper.start> is set to true
nifi.state.management.embedded.zookeeper.properties=./conf/zookeeper.properties

# H2 Settings
nifi.database.directory=./database_repository
nifi.h2.url.append=;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE

# Repository Encryption properties override individual repository implementation properties
nifi.repository.encryption.protocol.version=
nifi.repository.encryption.key.id=
nifi.repository.encryption.key.provider=
nifi.repository.encryption.key.provider.keystore.location=
nifi.repository.encryption.key.provider.keystore.password=

# FlowFile Repository
nifi.flowfile.repository.implementation=org.apache.nifi.controller.repository.WriteAheadFlowFileRepository
nifi.flowfile.repository.wal.implementation=org.apache.nifi.wali.SequentialAccessWriteAheadLog
nifi.flowfile.repository.directory=./flowfile_repository
nifi.flowfile.repository.checkpoint.interval=20 secs
nifi.flowfile.repository.always.sync=false
nifi.flowfile.repository.retain.orphaned.flowfiles=true

nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
nifi.queue.swap.threshold=20000

# Content Repository
nifi.content.repository.implementation=org.apache.nifi.controller.repository.FileSystemRepository
nifi.content.claim.max.appendable.size=50 KB
nifi.content.repository.directory.default=./content_repository
nifi.content.repository.archive.max.retention.period=1 days
nifi.content.repository.archive.max.usage.percentage=60%
nifi.content.repository.archive.enabled=false
nifi.content.repository.always.sync=false
nifi.content.viewer.url=../nifi-content-viewer/

# Provenance Repository Properties
nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository

# Persistent Provenance Repository Properties
nifi.provenance.repository.directory.default=./provenance_repository
nifi.provenance.repository.max.storage.time=1 day
nifi.provenance.repository.max.storage.size=1 GB
nifi.provenance.repository.rollover.time=10 mins
nifi.provenance.repository.rollover.size=100 MB
nifi.provenance.repository.query.threads=2
nifi.provenance.repository.index.threads=2
nifi.provenance.repository.compress.on.rollover=true
nifi.provenance.repository.always.sync=false
# Comma-separated list of fields. Fields that are not indexed will not be searchable. Valid fields are:
# EventType, FlowFileUUID, Filename, TransitURI, ProcessorID, AlternateIdentifierURI, Relationship, Details
nifi.provenance.repository.indexed.fields=EventType, FlowFileUUID, Filename, ProcessorID, Relationship
# FlowFile Attributes that should be indexed and made searchable.  Some examples to consider are filename, uuid, mime.type
nifi.provenance.repository.indexed.attributes=
# Large values for the shard size will result in more Java heap usage when searching the Provenance Repository
# but should provide better performance
nifi.provenance.repository.index.shard.size=500 MB
# Indicates the maximum length that a FlowFile attribute can be when retrieving a Provenance Event from
# the repository. If the length of any attribute exceeds this value, it will be truncated when the event is retrieved.
nifi.provenance.repository.max.attribute.length=65536
nifi.provenance.repository.concurrent.merge.threads=2


# Volatile Provenance Respository Properties
nifi.provenance.repository.buffer.size=100000

# Component and Node Status History Repository
nifi.components.status.repository.implementation=org.apache.nifi.controller.status.history.VolatileComponentStatusRepository

# Volatile Status History Repository Properties
nifi.components.status.repository.buffer.size=1440
nifi.components.status.snapshot.frequency=1 min

# QuestDB Status History Repository Properties
nifi.status.repository.questdb.persist.node.days=14
nifi.status.repository.questdb.persist.component.days=3
nifi.status.repository.questdb.persist.location=./status_repository

# Site to Site properties
nifi.remote.input.host=
nifi.remote.input.secure=true
nifi.remote.input.socket.port=10443
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs

# web properties #
#############################################

# For security, NiFi will present the UI on 127.0.0.1 and only be accessible through this loopback interface.
# Be aware that changing these properties may affect how your instance can be accessed without any restriction.
# We recommend configuring HTTPS instead. The administrators guide provides instructions on how to do this.

nifi.web.http.host=
nifi.web.http.network.interface.default=
nifi.web.https.host=
nifi.web.https.port=9443
nifi.web.https.network.interface.default=
nifi.web.https.application.protocols=http/1.1
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
nifi.web.max.header.size=16 KB
nifi.web.proxy.context.path=
nifi.web.proxy.host=
nifi.web.max.content.size=
nifi.web.max.requests.per.second=30000
nifi.web.max.access.token.requests.per.second=25
nifi.web.request.timeout=60 secs
nifi.web.request.ip.whitelist=
nifi.web.should.send.server.version=true
nifi.web.request.log.format=%{client}a - %u %t "%r" %s %O "%{Referer}i" "%{User-Agent}i"

# Include or Exclude TLS Cipher Suites for HTTPS
nifi.web.https.ciphersuites.include=
nifi.web.https.ciphersuites.exclude=

# security properties #
nifi.sensitive.props.key=11jba2NSNi50hDUN(
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=NIFI_PBKDF2_AES_GCM_256
nifi.sensitive.props.provider=BC
nifi.sensitive.props.additional.keys=

nifi.security.keystore=/opt/nifi/data/certs/keystore.jks
nifi.security.keystoreType=jks
nifi.security.keystorePasswd=notmyactualpassword
nifi.security.keyPasswd=notmyactualpassword
nifi.security.truststore=/opt/nifi/data/certs/truststore.jks
nifi.security.truststoreType=jks
nifi.security.truststorePasswd=notmyactualpassword

nifi.security.user.authorizer=managed-authorizer
nifi.security.allow.anonymous.authentication=false
nifi.security.user.login.identity.provider=
nifi.security.user.jws.key.rotation.period=PT1H
nifi.security.ocsp.responder.url=
nifi.security.ocsp.responder.certificate=

# OpenId Connect SSO Properties #
nifi.security.user.oidc.discovery.url=
nifi.security.user.oidc.connect.timeout=5 secs
nifi.security.user.oidc.read.timeout=5 secs
nifi.security.user.oidc.client.id=
nifi.security.user.oidc.client.secret=
nifi.security.user.oidc.preferred.jwsalgorithm=
nifi.security.user.oidc.additional.scopes=
nifi.security.user.oidc.claim.identifying.user=
nifi.security.user.oidc.fallback.claims.identifying.user=
nifi.security.user.oidc.truststore.strategy=JDK

# Apache Knox SSO Properties #
nifi.security.user.knox.url=
nifi.security.user.knox.publicKey=
nifi.security.user.knox.cookieName=hadoop-jwt
nifi.security.user.knox.audiences=

# SAML Properties #
nifi.security.user.saml.idp.metadata.url=https://fs.lgt.com/FederationMetadata/2007-06/FederationMetadata.xml
nifi.security.user.saml.sp.entity.id=data:devplatform:nifi:saml:sp
nifi.security.user.saml.identity.attribute.name=Name ID
nifi.security.user.saml.group.attribute.name=http://schemas.xmlsoap.org/claims/Group
nifi.security.user.saml.request.signing.enabled=false
nifi.security.user.saml.want.assertions.signed=true
nifi.security.user.saml.signature.algorithm=http://www.w3.org/2001/04/xmldsig-more#rsa-sha256
nifi.security.user.saml.authentication.expiration=12 hours
nifi.security.user.saml.single.logout.enabled=false
nifi.security.user.saml.http.client.truststore.strategy=JDK
nifi.security.user.saml.http.client.connect.timeout=30 secs
nifi.security.user.saml.http.client.read.timeout=30 secs
nifi.security.user.saml.message.logging.enabled=true

# Identity Mapping Properties #
# These properties allow normalizing user identities such that identities coming from different identity providers
# (certificates, LDAP, Kerberos) can be treated the same internally in NiFi. The following example demonstrates normalizing
# DNs from certificates and principals from Kerberos into a common identity string:
#
# nifi.security.identity.mapping.pattern.dn=^CN=(.*?), OU=(.*?), O=(.*?), L=(.*?), ST=(.*?), C=(.*?)$
# nifi.security.identity.mapping.value.dn=$1@$2
# nifi.security.identity.mapping.transform.dn=NONE
# nifi.security.identity.mapping.pattern.kerb=^(.*?)/instance@(.*?)$
# nifi.security.identity.mapping.value.kerb=$1@$2
# nifi.security.identity.mapping.transform.kerb=UPPER

# Group Mapping Properties #
# These properties allow normalizing group names coming from external sources like LDAP. The following example
# lowercases any group name.
#
# nifi.security.group.mapping.pattern.anygroup=^(.*)$
# nifi.security.group.mapping.value.anygroup=$1
# nifi.security.group.mapping.transform.anygroup=LOWER

# Listener Bootstrap properties #
# This property defines the port used to listen for communications from NiFi Bootstrap. If this property
# is missing, empty, or 0, a random ephemeral port is used.
nifi.listener.bootstrap.port=0

# cluster common properties (all nodes must have same values) #
nifi.cluster.protocol.heartbeat.interval=12 sec
nifi.cluster.protocol.heartbeat.missable.max=8
nifi.cluster.protocol.is.secure=true

# cluster node properties (only configure for cluster nodes) #
nifi.cluster.is.node=true
nifi.cluster.node.address=
nifi.cluster.node.protocol.port=6007
nifi.cluster.node.protocol.max.threads=50
nifi.cluster.node.event.history.size=25
nifi.cluster.node.connection.timeout=30 sec
nifi.cluster.node.read.timeout=60 sec
nifi.cluster.node.max.concurrent.requests=100
nifi.cluster.firewall.file=
nifi.cluster.flow.election.max.wait.time=30 sec
nifi.cluster.flow.election.max.candidates=

# cluster load balancing properties #
nifi.cluster.load.balance.host=
nifi.cluster.load.balance.port=6342
nifi.cluster.load.balance.connections.per.node=1
nifi.cluster.load.balance.max.thread.count=8
nifi.cluster.load.balance.comms.timeout=30 sec

# zookeeper properties, used for cluster management #
nifi.zookeeper.connect.string=nifi-zookeeper-svc:2181
nifi.zookeeper.connect.timeout=10 secs
nifi.zookeeper.session.timeout=10 secs
nifi.zookeeper.root.node=/nifi
nifi.zookeeper.client.secure=false
nifi.zookeeper.security.keystore=
nifi.zookeeper.security.keystoreType=
nifi.zookeeper.security.keystorePasswd=
nifi.zookeeper.security.truststore=
nifi.zookeeper.security.truststoreType=
nifi.zookeeper.security.truststorePasswd=
nifi.zookeeper.jute.maxbuffer=

# Zookeeper properties for the authentication scheme used when creating acls on znodes used for cluster management
# Values supported for nifi.zookeeper.auth.type are "default", which will apply world/anyone rights on znodes
# and "sasl" which will give rights to the sasl/kerberos identity used to authenticate the nifi node
# The identity is determined using the value in nifi.kerberos.service.principal and the removeHostFromPrincipal
# and removeRealmFromPrincipal values (which should align with the kerberos.removeHostFromPrincipal and kerberos.removeRealmFromPrincipal
# values configured on the zookeeper server).
nifi.zookeeper.auth.type=
nifi.zookeeper.kerberos.removeHostFromPrincipal=
nifi.zookeeper.kerberos.removeRealmFromPrincipal=

# kerberos #
nifi.kerberos.krb5.file=

# kerberos service principal #
nifi.kerberos.service.principal=
nifi.kerberos.service.keytab.location=

# kerberos spnego principal #
nifi.kerberos.spnego.principal=
nifi.kerberos.spnego.keytab.location=
nifi.kerberos.spnego.authentication.expiration=12 hours

# external properties files for variable registry
# supports a comma delimited list of file locations
nifi.variable.registry.properties=

# analytics properties #
nifi.analytics.predict.enabled=false
nifi.analytics.predict.interval=3 mins
nifi.analytics.query.interval=5 mins
nifi.analytics.connection.model.implementation=org.apache.nifi.controller.status.analytics.models.OrdinaryLeastSquares
nifi.analytics.connection.model.score.name=rSquared
nifi.analytics.connection.model.score.threshold=.90

# runtime monitoring properties
nifi.monitor.long.running.task.schedule=
nifi.monitor.long.running.task.threshold=

# Create automatic diagnostics when stopping/restarting NiFi.

# Enable automatic diagnostic at shutdown.
nifi.diagnostics.on.shutdown.enabled=false

# Include verbose diagnostic information.
nifi.diagnostics.on.shutdown.verbose=false

# The location of the diagnostics folder.
nifi.diagnostics.on.shutdown.directory=./diagnostics

# The maximum number of files permitted in the directory. If the limit is exceeded, the oldest files are deleted.
nifi.diagnostics.on.shutdown.max.filecount=10

# The diagnostics folder's maximum permitted size in bytes. If the limit is exceeded, the oldest files are deleted.
nifi.diagnostics.on.shutdown.max.directory.size=10 MB

# Performance tracking properties
## Specifies what percentage of the time we should track the amount of time processors are using CPU, reading from/writing to content repo, etc.
## This can be useful to understand which components are the most expensive and to understand where system bottlenecks may be occurring.
## The value must be in the range of 0 (inclusive) to 100 (inclusive). A larger value will produce more accurate results, while a smaller value may be
## less expensive to compute.
## Results can be obtained by running "nifi.sh diagnostics <filename>" and then inspecting the produced file.
nifi.performance.tracking.percentage=0

# NAR Provider Properties #
# These properties allow configuring one or more NAR providers. A NAR provider retrieves NARs from an external source
# and copies them to the directory specified by nifi.nar.library.autoload.directory.
#
# Each NAR provider property follows the format:
#  nifi.nar.library.provider.<identifier>.<property-name>
#
# Each NAR provider must have at least one property named "implementation".
#
# Example HDFS NAR Provider:
#   nifi.nar.library.provider.hdfs.implementation=org.apache.nifi.flow.resource.hadoop.HDFSExternalResourceProvider
#   nifi.nar.library.provider.hdfs.resources=/path/to/core-site.xml,/path/to/hdfs-site.xml
#   nifi.nar.library.provider.hdfs.storage.location=hdfs://hdfs-location
#   nifi.nar.library.provider.hdfs.source.directory=/nars
#   nifi.nar.library.provider.hdfs.kerberos.principal=nifi@NIFI.COM
#   nifi.nar.library.provider.hdfs.kerberos.keytab=/path/to/nifi.keytab
#   nifi.nar.library.provider.hdfs.kerberos.password=
#
# Example NiFi Registry NAR Provider:
#   nifi.nar.library.provider.nifi-registry.implementation=org.apache.nifi.registry.extension.NiFiRegistryNarProvider
#   nifi.nar.library.provider.nifi-registry.url=http://localhost:18080

 

I am running nifi 2.0.0-M4 on OpenShift. My two node cluster comes up fine but as soon as I try to enable the python-api by setting nifi.python.command=python3 it keeps crashing. Here the relevant log section:

 

mkdir: created directory '/opt/nifi/nifi-current/work'
mkdir: created directory '/opt/nifi/nifi-current/work/python'
updating nifi.cluster.node.address in /opt/nifi/nifi-current/conf/nifi.properties
updating nifi.remote.input.host in /opt/nifi/nifi-current/conf/nifi.properties
updating nifi.web.https.host in /opt/nifi/nifi-current/conf/nifi.properties
updating nifi.web.proxy.host in /opt/nifi/nifi-current/conf/nifi.properties

Java home: /usr/lib/jvm/jre
NiFi home: /opt/nifi/nifi-current

Bootstrap Config File: /opt/nifi/nifi-current/conf/bootstrap.conf

Login Identity Providers Processed [/opt/nifi/nifi-current/./conf/login-identity-providers.xml]

Certificate was added to keystore
Certificate was added to keystore
NiFi running with PID 118.

Java home: /usr/lib/jvm/jre
NiFi home: /opt/nifi/nifi-current

Bootstrap Config File: /opt/nifi/nifi-current/conf/bootstrap.conf

07:28:09,319 |-INFO in ch.qos.logback.classic.LoggerContext[default] - This is logback-classic version 1.5.6
07:28:09,320 |-INFO in ch.qos.logback.classic.util.ContextInitializer@2aa5fe93 - No custom configurators were discovered as a service.
07:28:09,320 |-INFO in ch.qos.logback.classic.util.ContextInitializer@2aa5fe93 - Trying to configure with ch.qos.logback.classic.joran.SerializedModelConfigurator
07:28:09,321 |-INFO in ch.qos.logback.classic.util.ContextInitializer@2aa5fe93 - Constructed configurator of type class ch.qos.logback.classic.joran.SerializedModelConfigurator
07:28:09,326 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.scmo]
07:28:09,326 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.scmo]
07:28:09,334 |-INFO in ch.qos.logback.classic.util.ContextInitializer@2aa5fe93 - ch.qos.logback.classic.joran.SerializedModelConfigurator.configure() call lasted 6 milliseconds. ExecutionStatus=INVOKE_NEXT_IF_ANY
07:28:09,334 |-INFO in ch.qos.logback.classic.util.ContextInitializer@2aa5fe93 - Trying to configure with ch.qos.logback.classic.util.DefaultJoranConfigurator
07:28:09,334 |-INFO in ch.qos.logback.classic.util.ContextInitializer@2aa5fe93 - Constructed configurator of type class ch.qos.logback.classic.util.DefaultJoranConfigurator
07:28:09,335 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
07:28:09,339 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [file:/opt/nifi/nifi-current/conf/logback.xml]
07:28:09,458 |-INFO in ch.qos.logback.classic.model.processor.ConfigurationModelHandlerFull - Registering a new ReconfigureOnChangeTask ReconfigureOnChangeTask(born:1722324489456)
07:28:09,459 |-INFO in ch.qos.logback.classic.model.processor.ConfigurationModelHandlerFull - Will scan for changes in [file:/opt/nifi/nifi-current/conf/logback.xml] 
07:28:09,459 |-INFO in ch.qos.logback.classic.model.processor.ConfigurationModelHandlerFull - Setting ReconfigureOnChangeTask scanning period to 30 seconds
07:28:09,469 |-WARN in ch.qos.logback.core.model.processor.ShutdownHookModelHandler - ch.qos.logback.core.hook.DelayingShutdownHook was renamed as ch.qos.logback.core.hook.DefaultShutdownHook
07:28:09,469 |-WARN in ch.qos.logback.core.model.processor.ShutdownHookModelHandler - Please use the new class name
07:28:09,469 |-INFO in ch.qos.logback.core.model.processor.ShutdownHookModelHandler - About to instantiate shutdown hook of type [ch.qos.logback.core.hook.DefaultShutdownHook]
07:28:09,469 |-INFO in ch.qos.logback.core.model.processor.ShutdownHookModelHandler - Registering shutdown hook with JVM runtime.
07:28:09,470 |-INFO in ch.qos.logback.classic.model.processor.LoggerContextListenerModelHandler - Adding LoggerContextListener of type [ch.qos.logback.classic.jul.LevelChangePropagator] to the object stack
07:28:09,482 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating DEBUG level on Logger[ROOT] onto the JUL framework
07:28:09,482 |-INFO in ch.qos.logback.classic.model.processor.LoggerContextListenerModelHandler - Starting LoggerContextListener
07:28:09,484 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - Processing appender named [APP_FILE]
07:28:09,484 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
07:28:09,496 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@1524126153 - setting totalSizeCap to 500 MB
07:28:09,499 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@1524126153 - Archive files will be limited to [100 MB] each.
07:28:09,514 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@1524126153 - No compression will be used
07:28:09,515 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@1524126153 - Will use the pattern /opt/nifi/nifi-current/logs/nifi-app_%d.%i.log for the active file
07:28:09,544 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6156496 - The date pattern is 'yyyy-MM-dd' from file name pattern '/opt/nifi/nifi-current/logs/nifi-app_%d.%i.log'.
07:28:09,544 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6156496 - Roll-over at midnight.
07:28:09,550 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@6156496 - Setting initial period to 2024-07-30T07:27:55.654Z
07:28:09,552 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@1524126153 - Cleaning on start up
07:28:09,553 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - first clean up after appender initialization
07:28:09,553 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - Multiple periods, i.e. 32 periods, seem to have elapsed. This is expected at application start.
07:28:09,561 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - No removal attempts were made.
07:28:09,568 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - Active log file name: /opt/nifi/nifi-current/logs/nifi-app.log
07:28:09,569 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[APP_FILE] - File property is set to [/opt/nifi/nifi-current/logs/nifi-app.log]
07:28:09,569 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - Processing appender named [USER_FILE]
07:28:09,569 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
07:28:09,570 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@63001505 - setting totalSizeCap to 200 MB
07:28:09,570 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@63001505 - Archive files will be limited to [100 MB] each.
07:28:09,570 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@63001505 - No compression will be used
07:28:09,570 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@63001505 - Will use the pattern /opt/nifi/nifi-current/logs/nifi-user_%d.%i.log for the active file
07:28:09,571 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@b62fe6d - The date pattern is 'yyyy-MM-dd' from file name pattern '/opt/nifi/nifi-current/logs/nifi-user_%d.%i.log'.
07:28:09,571 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@b62fe6d - Roll-over at midnight.
07:28:09,571 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@b62fe6d - Setting initial period to 2024-07-30T07:27:54.768Z
07:28:09,572 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@63001505 - Cleaning on start up
07:28:09,572 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - first clean up after appender initialization
07:28:09,572 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - Multiple periods, i.e. 32 periods, seem to have elapsed. This is expected at application start.
07:28:09,573 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[USER_FILE] - Active log file name: /opt/nifi/nifi-current/logs/nifi-user.log
07:28:09,573 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[USER_FILE] - File property is set to [/opt/nifi/nifi-current/logs/nifi-user.log]
07:28:09,573 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - Processing appender named [REQUEST_FILE]
07:28:09,573 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
07:28:09,573 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@330084561 - setting totalSizeCap to 200 MB
07:28:09,573 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@330084561 - Archive files will be limited to [100 MB] each.
07:28:09,574 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@330084561 - No compression will be used
07:28:09,574 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@330084561 - Will use the pattern /opt/nifi/nifi-current/logs/nifi-request_%d.%i.log for the active file
07:28:09,574 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@3e3047e6 - The date pattern is 'yyyy-MM-dd' from file name pattern '/opt/nifi/nifi-current/logs/nifi-request_%d.%i.log'.
07:28:09,574 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@3e3047e6 - Roll-over at midnight.
07:28:09,574 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@3e3047e6 - Setting initial period to 2024-07-30T07:02:54.353Z
07:28:09,575 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@330084561 - Cleaning on start up
07:28:09,575 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - first clean up after appender initialization
07:28:09,575 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - Multiple periods, i.e. 32 periods, seem to have elapsed. This is expected at application start.
07:28:09,575 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[REQUEST_FILE] - Active log file name: /opt/nifi/nifi-current/logs/nifi-request.log
07:28:09,575 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[REQUEST_FILE] - File property is set to [/opt/nifi/nifi-current/logs/nifi-request.log]
07:28:09,575 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - Processing appender named [BOOTSTRAP_FILE]
07:28:09,575 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
07:28:09,576 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@937773018 - setting totalSizeCap to 100 MB
07:28:09,576 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@937773018 - Archive files will be limited to [50 MB] each.
07:28:09,576 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@937773018 - No compression will be used
07:28:09,576 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@937773018 - Will use the pattern /opt/nifi/nifi-current/logs/nifi-bootstrap_%d.%i.log for the active file
07:28:09,577 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@2b6856dd - The date pattern is 'yyyy-MM-dd' from file name pattern '/opt/nifi/nifi-current/logs/nifi-bootstrap_%d.%i.log'.
07:28:09,577 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@2b6856dd - Roll-over at midnight.
07:28:09,577 |-INFO in ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP@2b6856dd - Setting initial period to 2024-07-30T07:27:55.654Z
07:28:09,577 |-INFO in c.q.l.core.rolling.SizeAndTimeBasedRollingPolicy@937773018 - Cleaning on start up
07:28:09,577 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - first clean up after appender initialization
07:28:09,578 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - Multiple periods, i.e. 32 periods, seem to have elapsed. This is expected at application start.
07:28:09,578 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - No removal attempts were made.
07:28:09,578 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[BOOTSTRAP_FILE] - Active log file name: /opt/nifi/nifi-current/logs/nifi-bootstrap.log
07:28:09,578 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[BOOTSTRAP_FILE] - File property is set to [/opt/nifi/nifi-current/logs/nifi-bootstrap.log]
07:28:09,578 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - Processing appender named [CONSOLE]
07:28:09,578 |-INFO in ch.qos.logback.core.model.processor.AppenderModelHandler - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
07:28:09,580 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - No removal attempts were made.
07:28:09,581 |-INFO in c.q.l.core.rolling.helper.TimeBasedArchiveRemover - No removal attempts were made.
07:28:09,582 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi] to INFO
07:28:09,582 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi] onto the JUL framework
07:28:09,582 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.processors] to WARN
07:28:09,582 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[org.apache.nifi.processors] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.processors.standard.LogAttribute] to WARN
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[org.apache.nifi.processors.standard.LogAttribute] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.processors.standard.LogMessage] to WARN
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[org.apache.nifi.processors.standard.LogMessage] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.controller.repository.StandardProcessSession] to WARN
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[org.apache.nifi.controller.repository.StandardProcessSession] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.zookeeper.ClientCnxn] to ERROR
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.apache.zookeeper.ClientCnxn] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.zookeeper.server.NIOServerCnxn] to ERROR
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.apache.zookeeper.server.NIOServerCnxn] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.zookeeper.server.NIOServerCnxnFactory] to ERROR
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.apache.zookeeper.server.NIOServerCnxnFactory] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.zookeeper.server.NettyServerCnxnFactory] to ERROR
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.apache.zookeeper.server.NettyServerCnxnFactory] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.zookeeper.server.quorum] to ERROR
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.apache.zookeeper.server.quorum] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.zookeeper.ZooKeeper] to ERROR
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.apache.zookeeper.ZooKeeper] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.zookeeper.server.PrepRequestProcessor] to ERROR
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.apache.zookeeper.server.PrepRequestProcessor] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.controller.reporting.LogComponentStatuses] to ERROR
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.apache.nifi.controller.reporting.LogComponentStatuses] onto the JUL framework
07:28:09,583 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.calcite.runtime.CalciteException] to OFF
07:28:09,583 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating OFF level on Logger[org.apache.calcite.runtime.CalciteException] onto the JUL framework
07:28:09,584 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.curator.framework.recipes.leader.LeaderSelector] to OFF
07:28:09,584 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating OFF level on Logger[org.apache.curator.framework.recipes.leader.LeaderSelector] onto the JUL framework
07:28:09,584 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.curator.ConnectionState] to OFF
07:28:09,584 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating OFF level on Logger[org.apache.curator.ConnectionState] onto the JUL framework
07:28:09,584 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.cluster] to INFO
07:28:09,584 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi.cluster] onto the JUL framework
07:28:09,584 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.server.JettyServer] to INFO
07:28:09,584 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi.server.JettyServer] onto the JUL framework
07:28:09,584 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.eclipse.jetty] to INFO
07:28:09,584 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.eclipse.jetty] onto the JUL framework
07:28:09,584 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.springframework] to ERROR
07:28:09,584 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.springframework] onto the JUL framework
07:28:09,584 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.springframework.security] to INFO
07:28:09,584 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.springframework.security] onto the JUL framework
07:28:09,584 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.glassfish.jersey.internal.Errors] to ERROR
07:28:09,584 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.glassfish.jersey.internal.Errors] onto the JUL framework
07:28:09,584 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.eclipse.jetty.annotations.AnnotationParser] to ERROR
07:28:09,584 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.eclipse.jetty.annotations.AnnotationParser] onto the JUL framework
07:28:09,584 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [net.schmizz.sshj] to WARN
07:28:09,584 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[net.schmizz.sshj] onto the JUL framework
07:28:09,584 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [com.hierynomus.sshj] to WARN
07:28:09,584 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[com.hierynomus.sshj] onto the JUL framework
07:28:09,585 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [com.hierynomus.smbj] to WARN
07:28:09,585 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[com.hierynomus.smbj] onto the JUL framework
07:28:09,585 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [com.amazonaws.services.kinesis] to WARN
07:28:09,585 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[com.amazonaws.services.kinesis] onto the JUL framework
07:28:09,585 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.atlas] to WARN
07:28:09,585 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[org.apache.atlas] onto the JUL framework
07:28:09,585 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.web.security.requests] to INFO
07:28:09,585 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi.web.security.requests] onto the JUL framework
07:28:09,588 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.apache.nifi.web.security.requests] to false
07:28:09,589 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [APP_FILE] to Logger[org.apache.nifi.web.security.requests]
07:28:09,589 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [CONSOLE] to Logger[org.apache.nifi.web.security.requests]
07:28:09,589 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.web.security] to INFO
07:28:09,589 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi.web.security] onto the JUL framework
07:28:09,589 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.apache.nifi.web.security] to false
07:28:09,589 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [USER_FILE] to Logger[org.apache.nifi.web.security]
07:28:09,589 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.web.api.config] to INFO
07:28:09,589 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi.web.api.config] onto the JUL framework
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.apache.nifi.web.api.config] to false
07:28:09,590 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [USER_FILE] to Logger[org.apache.nifi.web.api.config]
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.authorization] to INFO
07:28:09,590 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi.authorization] onto the JUL framework
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.apache.nifi.authorization] to false
07:28:09,590 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [USER_FILE] to Logger[org.apache.nifi.authorization]
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.cluster.authorization] to INFO
07:28:09,590 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi.cluster.authorization] onto the JUL framework
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.apache.nifi.cluster.authorization] to false
07:28:09,590 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [USER_FILE] to Logger[org.apache.nifi.cluster.authorization]
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.web.api.AccessResource] to INFO
07:28:09,590 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi.web.api.AccessResource] onto the JUL framework
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.apache.nifi.web.api.AccessResource] to false
07:28:09,590 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [USER_FILE] to Logger[org.apache.nifi.web.api.AccessResource]
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.opensaml] to WARN
07:28:09,590 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[org.opensaml] onto the JUL framework
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.opensaml] to false
07:28:09,590 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [USER_FILE] to Logger[org.opensaml]
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.web.server.RequestLog] to INFO
07:28:09,590 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi.web.server.RequestLog] onto the JUL framework
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.apache.nifi.web.server.RequestLog] to false
07:28:09,590 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [REQUEST_FILE] to Logger[org.apache.nifi.web.server.RequestLog]
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.bootstrap] to INFO
07:28:09,590 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi.bootstrap] onto the JUL framework
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.apache.nifi.bootstrap] to true
07:28:09,590 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [BOOTSTRAP_FILE] to Logger[org.apache.nifi.bootstrap]
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.bootstrap.Command] to INFO
07:28:09,590 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating INFO level on Logger[org.apache.nifi.bootstrap.Command] onto the JUL framework
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.apache.nifi.bootstrap.Command] to true
07:28:09,590 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [BOOTSTRAP_FILE] to Logger[org.apache.nifi.bootstrap.Command]
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.StdOut] to WARN
07:28:09,590 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[org.apache.nifi.StdOut] onto the JUL framework
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.apache.nifi.StdOut] to true
07:28:09,590 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [BOOTSTRAP_FILE] to Logger[org.apache.nifi.StdOut]
07:28:09,590 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting level of logger [org.apache.nifi.StdErr] to ERROR
07:28:09,591 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating ERROR level on Logger[org.apache.nifi.StdErr] onto the JUL framework
07:28:09,591 |-INFO in ch.qos.logback.classic.model.processor.LoggerModelHandler - Setting additivity of logger [org.apache.nifi.StdErr] to true
07:28:09,591 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [BOOTSTRAP_FILE] to Logger[org.apache.nifi.StdErr]
07:28:09,591 |-INFO in ch.qos.logback.classic.model.processor.RootLoggerModelHandler - Setting level of ROOT logger to WARN
07:28:09,591 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@5c1a8622 - Propagating WARN level on Logger[ROOT] onto the JUL framework
07:28:09,591 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [CONSOLE] to Logger[ROOT]
07:28:09,591 |-INFO in ch.qos.logback.core.model.processor.AppenderRefModelHandler - Attaching appender named [APP_FILE] to Logger[ROOT]
07:28:09,591 |-INFO in ch.qos.logback.core.model.processor.DefaultProcessor@5db45159 - End of configuration.
07:28:09,592 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@6107227e - Registering current configuration as safe fallback point
07:28:09,592 |-INFO in ch.qos.logback.classic.util.ContextInitializer@2aa5fe93 - ch.qos.logback.classic.util.DefaultJoranConfigurator.configure() call lasted 258 milliseconds. ExecutionStatus=DO_NOT_INVOKE_NEXT_IF_ANY

2024-07-30 07:28:09,613 [main] WARN  org.apache.nifi.bootstrap.process.RuntimeValidatorExecutor - -Runtime Configuration [AvailableLocalPorts] validation failed: Local Ports [28231] less than recommended [55000] according to [/proc/sys/net/ipv4/ip_local_port_range]
2024-07-30 07:28:09,615 [main] WARN  org.apache.nifi.bootstrap.process.RuntimeValidatorExecutor - -Runtime Configuration [Swappiness] validation failed: Swappiness [30] more than recommended [0] according to [/proc/sys/vm/swappiness]
2024-07-30 07:28:09,616 [main] WARN  org.apache.nifi.bootstrap.process.RuntimeValidatorExecutor - -Runtime Configuration [SocketTimedWaitDuration] validation failed: TCP Socket Wait [120 seconds] more than recommended [1 seconds] according to [/proc/sys/net/netfilter/nf_conntrack_tcp_timeout_time_wait]
2024-07-30 07:28:09,721 [main] INFO  org.apache.nifi.bootstrap.RunNiFi - -{javaVersion=21.0.4, availableProcessors=4, maxOpenFileDescriptors=1048576, totalPhysicalMemoryMB=20480}
2024-07-30 07:28:09,749 [main] INFO  org.apache.nifi.bootstrap.Command - -Starting Apache NiFi...
2024-07-30 07:28:09,750 [main] INFO  org.apache.nifi.bootstrap.Command - -Working Directory: /opt/nifi/nifi-current
2024-07-30 07:28:09,750 [main] INFO  org.apache.nifi.bootstrap.Command - -Command: /usr/lib/jvm/jre/bin/java -classpath /opt/nifi/nifi-current/./conf:/opt/nifi/nifi-current/./lib/jcl-over-slf4j-2.0.13.jar:/opt/nifi/nifi-current/./lib/jul-to-slf4j-2.0.13.jar:/opt/nifi/nifi-current/./lib/log4j-over-slf4j-2.0.13.jar:/opt/nifi/nifi-current/./lib/logback-classic-1.5.6.jar:/opt/nifi/nifi-current/./lib/logback-core-1.5.6.jar:/opt/nifi/nifi-current/./lib/nifi-api-2.0.0-M4.jar:/opt/nifi/nifi-current/./lib/nifi-framework-api-2.0.0-M4.jar:/opt/nifi/nifi-current/./lib/nifi-nar-utils-2.0.0-M4.jar:/opt/nifi/nifi-current/./lib/nifi-per-process-group-logging-2.0.0-M4.jar:/opt/nifi/nifi-current/./lib/nifi-properties-2.0.0-M4.jar:/opt/nifi/nifi-current/./lib/nifi-property-utils-2.0.0-M4.jar:/opt/nifi/nifi-current/./lib/nifi-python-framework-api-2.0.0-M4.jar:/opt/nifi/nifi-current/./lib/nifi-runtime-2.0.0-M4.jar:/opt/nifi/nifi-current/./lib/nifi-server-api-2.0.0-M4.jar:/opt/nifi/nifi-current/./lib/nifi-stateless-api-2.0.0-M4.jar:/opt/nifi/nifi-current/./lib/slf4j-api-2.0.13.jar:/opt/nifi/nifi-current/./lib/ojdbc8-18.0.0.0.jar -Dorg.apache.jasper.compiler.disablejsr199=true -XX:MaxRAMPercentage=70 -Djava.security.egd=file:/dev/urandom -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Djava.protocol.handler.pkgs=sun.net.www.protocol -XX:+UseG1GC -Dnifi.properties.file.path=/opt/nifi/nifi-current/./conf/nifi.properties -Dnifi.bootstrap.listen.port=41379 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=/opt/nifi/nifi-current/logs org.apache.nifi.NiFi 
2024-07-30 07:28:09,762 [main] INFO  org.apache.nifi.bootstrap.Command - -Application Process [157] launched
2024-07-30 07:28:10,173 [NiFi Bootstrap Command Listener] INFO  org.apache.nifi.bootstrap.RunNiFi - -Apache NiFi now running and listening for Bootstrap requests on port 38023
2024-07-30 07:28:20,769 [NiFi logging handler] ERROR org.apache.nifi.StdErr - -WARNING: A restricted method in java.lang.foreign.Linker has been called
2024-07-30 07:28:20,769 [NiFi logging handler] ERROR org.apache.nifi.StdErr - -WARNING: java.lang.foreign.Linker::downcallHandle has been called by the unnamed module
2024-07-30 07:28:20,770 [NiFi logging handler] ERROR org.apache.nifi.StdErr - -WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for this module
2024-07-30 07:28:20,770 [NiFi logging handler] ERROR org.apache.nifi.StdErr - -
2024-07-30 07:28:21,536 [NiFi logging handler] ERROR org.apache.nifi.StdErr - -Failed to start web server: Error creating bean with name 'frameworkClusterConfiguration': Unsatisfied dependency expressed through method 'setFlowController' parameter 0: Error creating bean with name 'flowController' defined in class path resource [org/apache/nifi/framework/configuration/FlowControllerConfiguration.class]: Failed to instantiate [org.apache.nifi.controller.FlowController]: Factory method 'flowController' threw exception with message: null
2024-07-30 07:28:21,536 [NiFi logging handler] ERROR org.apache.nifi.StdErr - -Shutting down...
2024-07-30 07:28:22,766 [main] INFO  org.apache.nifi.bootstrap.RunNiFi - -NiFi never started. Will not restart NiFi

 

I checked that my java and my python as required:

 

sh-4.4$ python3 --version
Python 3.9.19
sh-4.4$ python3 -m venv
usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear] [--upgrade] [--without-pip] [--prompt PROMPT]
            [--upgrade-deps]
            ENV_DIR [ENV_DIR ...]
venv: error: the following arguments are required: ENV_DIR
sh-4.4$ java --version
openjdk 21.0.4 2024-07-16 LTS
OpenJDK Runtime Environment (Red_Hat-21.0.4.0.7-1) (build 21.0.4+7-LTS)
OpenJDK 64-Bit Server VM (Red_Hat-21.0.4.0.7-1) (build 21.0.4+7-LTS, mixed mode, sharing)

 

What I think is weird that there are no python logs in $NIFI_HOME/logs. Also $NIFI_HOME/work/python is empty as well.  Here is my nifi.properties file:

 

 

1 ACCEPTED SOLUTION

avatar
New Contributor

I asked in the slack channel of apache nifi. So basically it's all in the stacktrace: org.apache.nifi.controller.FlowController.createPythonBridge(FlowController.java:865. The line on the source code: https://github.com/apache/nifi/blob/rel/nifi-2.0.0-M4/nifi-framework-bundle/nifi-framework/nifi-fram....  Here as a snippet at your convenience: 

final File pythonWorkingDirectory = new File(nifiProperties.getProperty(NiFiProperties.PYTHON_WORKING_DIRECTORY));

so because we have a bit of a custom setup we did get all the default python settings as outlined here: https://nifi.apache.org/documentation/nifi-2.0.0-M1/html/administration-guide.html#python_configurat...

After setting all of those: 

 nifi.python.command=python3
 nifi.python.framework.source.directory=./python/framework
 nifi.python.extensions.source.directory.default=./python/extensions
 nifi.python.working.directory=./work/python 
 nifi.python.logs.directory=./logs 
 nifi.python.max.processes.per.extension.type=10 
 nifi.python.max.processes=100

it then started again and I could import the processor

View solution in original post

1 REPLY 1

avatar
New Contributor

I asked in the slack channel of apache nifi. So basically it's all in the stacktrace: org.apache.nifi.controller.FlowController.createPythonBridge(FlowController.java:865. The line on the source code: https://github.com/apache/nifi/blob/rel/nifi-2.0.0-M4/nifi-framework-bundle/nifi-framework/nifi-fram....  Here as a snippet at your convenience: 

final File pythonWorkingDirectory = new File(nifiProperties.getProperty(NiFiProperties.PYTHON_WORKING_DIRECTORY));

so because we have a bit of a custom setup we did get all the default python settings as outlined here: https://nifi.apache.org/documentation/nifi-2.0.0-M1/html/administration-guide.html#python_configurat...

After setting all of those: 

 nifi.python.command=python3
 nifi.python.framework.source.directory=./python/framework
 nifi.python.extensions.source.directory.default=./python/extensions
 nifi.python.working.directory=./work/python 
 nifi.python.logs.directory=./logs 
 nifi.python.max.processes.per.extension.type=10 
 nifi.python.max.processes=100

it then started again and I could import the processor