Apache Nifi CrashLoopBackOff on single node k8 cluster on docker desktop

KRISHANKUMAR — Tue, 11 Feb 2025 05:58:06 GMT

i am facing challenge to deploy the apache nifi on single node k8 cluster in docker desktop. Here is my nifi-config.yml and nifi-deployment.yml file with following error message.

nifi-config.yml

apiVersion: v1

kind: ConfigMap

metadata:

data:

nifi.properties: |

nifi.web.http.host=0.0.0.0

nifi.web.http.port=8080

# Site to Site properties (if needed)

# nifi.remote.input.host=...

# nifi.remote.input.secure=...

# nifi.remote.input.socket.port=...

# nifi.remote.input.http.enabled=...

# nifi.remote.input.http.transaction.ttl=30 sec

# HTTPS properties (if applicable)

# nifi.web.https.host=...

# nifi.web.https.port=...

# nifi.web.https.network.interface.default=...

# nifi.web.https.application.protocols=h2 http/1.1

# TLS Cipher Suites (if applicable)

# nifi.web.https.ciphersuites.include=...

# nifi.web.https.ciphersuites.exclude=...

nifi-deployment.yml

apiVersion: apps/v1

kind: Deployment

metadata:

labels:

app: nifi

spec:

replicas: 1

selector:

matchLabels:

app: nifi

template:

metadata:

labels:

app: nifi

spec:

# Init Container 1: Copy the entire default configuration directory from the image

initContainers:

- name: copy-default-config

image: apache/nifi:latest

command:

- sh

- -c

- |

cp -R /opt/nifi/nifi-current/conf/* /writable/

volumeMounts:

- name: writable-config

mountPath: /writable

# Init Container 2: Override the default nifi.properties with our custom version

- name: override-nifi-properties

image: busybox:latest

command:

- sh

- -c

- |

cp /config/nifi.properties /writable/nifi.properties

volumeMounts:

- name: config-volume

mountPath: /config

- name: writable-config

mountPath: /writable

# Init Container 3: Update bootstrap.conf to force foreground mode

- name: update-bootstrap-conf

image: busybox:latest

command:

- sh

- -c

- |

if [ -f /writable/bootstrap.conf ]; then

# Replace the line even if commented (handles both commented and uncommented lines)

sed -i 's/^#\?nifi\.bootstrap\.runInForeground=.*/nifi.bootstrap.runInForeground=true/' /writable/bootstrap.conf

else

echo "bootstrap.conf not found in /writable!"

exit 1

volumeMounts:

- name: writable-config

mountPath: /writable

# Init Container 4: Fix permissions on the entire configuration directory so that files are owned by UID 1000 (the nifi user)

- name: fix-permissions

image: busybox:latest

command:

- sh

- -c

- |

chown -R 1000:1000 /writable

volumeMounts:

- name: writable-config

mountPath: /writable

containers:

- name: nifi

image: apache/nifi:latest

ports:

- containerPort: 8080

env:

- name: NIFI_WEB_HTTP_PORT

value: "8080"

# Mount the complete configuration directory over NiFi’s conf directory

volumeMounts:

- name: writable-config

mountPath: /opt/nifi/nifi-current/conf

volumes:

- name: writable-config

emptyDir: {}

- name: config-volume

configMap:

kubectl describe pod

kubectl describe pod nifi-deployment-67f49c6cd5-ntc48
Name: nifi-deployment-67f49c6cd5-ntc48
Namespace: default
Priority: 0
Service Account: default
Node: docker-desktop/192.168.65.3
Start Time: Tue, 11 Feb 2025 11:23:07 +0530
Labels: app=nifi
pod-template-hash=67f49c6cd5
Annotations: <none>
Status: Running
IP: 10.1.1.102
IPs:
IP: 10.1.1.102
Controlled By: ReplicaSet/nifi-deployment-67f49c6cd5
Init Containers:
copy-default-config:
Container ID: docker://72e35257e520eaa5f37a7c2e0583cdd5f8991d35617f6d4407ae600d9293238c
Image: apache/nifi:latest
Image ID: docker-pullable://apache/nifi@sha256:6f8de66ca2f8eae07588508be7c40559077f646abbeb7472de7d855e38817b52
Port: <none>
Host Port: <none>
Command:
sh
-c
cp -R /opt/nifi/nifi-current/conf/* /writable/

State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 11 Feb 2025 11:23:12 +0530
Finished: Tue, 11 Feb 2025 11:23:13 +0530
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zh25w (ro)
/writable from writable-config (rw)
override-nifi-properties:
Container ID: docker://1a4ba0873de81c5e87616ed19b1da1119be5bb4d6c16dbb772efa156721a888a
Image: busybox:latest
Image ID: docker-pullable://busybox@sha256:a5d0ce49aa801d475da48f8cb163c354ab95cab073cd3c138bd458fc8257fbf1
Port: <none>
Host Port: <none>
Command:
sh
-c
cp /config/nifi.properties /writable/nifi.properties

State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 11 Feb 2025 11:23:16 +0530
Finished: Tue, 11 Feb 2025 11:23:16 +0530
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/config from config-volume (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zh25w (ro)
/writable from writable-config (rw)
update-bootstrap-conf:
Container ID: docker://b92faf93707dafa2b9a1e8de95ac097227453128064da3cb7d71cfb513966a18
Image: busybox:latest
Image ID: docker-pullable://busybox@sha256:a5d0ce49aa801d475da48f8cb163c354ab95cab073cd3c138bd458fc8257fbf1
Port: <none>
Host Port: <none>
Command:
sh
-c
if [ -f /writable/bootstrap.conf ]; then
# Replace the line even if commented (handles both commented and uncommented lines)
sed -i 's/^#\?nifi\.bootstrap\.runInForeground=.*/nifi.bootstrap.runInForeground=true/' /writable/bootstrap.conf
else
echo "bootstrap.conf not found in /writable!"
exit 1
fi

State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 11 Feb 2025 11:23:21 +0530
Finished: Tue, 11 Feb 2025 11:23:21 +0530
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zh25w (ro)
/writable from writable-config (rw)
fix-permissions:
Container ID: docker://c4ad5c183ec1d92609ac104c41419f7206da9a8258d72af9235ed8efc757a133
Image: busybox:latest
Image ID: docker-pullable://busybox@sha256:a5d0ce49aa801d475da48f8cb163c354ab95cab073cd3c138bd458fc8257fbf1
Port: <none>
Host Port: <none>
Command:
sh
-c
chown -R 1000:1000 /writable

State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 11 Feb 2025 11:23:24 +0530
Finished: Tue, 11 Feb 2025 11:23:24 +0530
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zh25w (ro)
/writable from writable-config (rw)
Containers:
nifi:
Container ID: docker://8e137edcf6f9be8e54b96930db92d6c3385829ac4a6508eed80a8f8fde90f7b3
Image: apache/nifi:latest
Image ID: docker-pullable://apache/nifi@sha256:6f8de66ca2f8eae07588508be7c40559077f646abbeb7472de7d855e38817b52
Port: 8080/TCP
Host Port: 0/TCP
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 11 Feb 2025 11:24:26 +0530
Finished: Tue, 11 Feb 2025 11:24:27 +0530
Last State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 11 Feb 2025 11:23:50 +0530
Finished: Tue, 11 Feb 2025 11:23:52 +0530
Ready: False
Restart Count: 3
Environment:
NIFI_WEB_HTTP_PORT: 8080
Mounts:
/opt/nifi/nifi-current/conf from writable-config (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zh25w (ro)
Conditions:
Type Status
PodReadyToStartContainers True
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
writable-config:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
config-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: nifi-config
Optional: false
kube-api-access-zh25w:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 87s default-scheduler Successfully assigned default/nifi-deployment-67f49c6cd5-ntc48 to docker-desktop
Normal Pulling 87s kubelet Pulling image "apache/nifi:latest"
Normal Pulled 83s kubelet Successfully pulled image "apache/nifi:latest" in 4.098s (4.098s including waiting). Image size: 1575137977 bytes.
Normal Created 83s kubelet Created container copy-default-config
Normal Started 82s kubelet Started container copy-default-config
Normal Pulling 82s kubelet Pulling image "busybox:latest"
Normal Pulled 79s kubelet Successfully pulled image "busybox:latest" in 2.942s (2.942s including waiting). Image size: 4269694 bytes.
Normal Created 79s kubelet Created container override-nifi-properties
Normal Started 79s kubelet Started container override-nifi-properties
Normal Pulling 78s kubelet Pulling image "busybox:latest"
Normal Pulled 74s kubelet Successfully pulled image "busybox:latest" in 3.368s (3.368s including waiting). Image size: 4269694 bytes.
Normal Created 74s kubelet Created container update-bootstrap-conf
Normal Started 74s kubelet Started container update-bootstrap-conf
Normal Pulling 74s kubelet Pulling image "busybox:latest"
Normal Pulled 71s kubelet Successfully pulled image "busybox:latest" in 3.034s (3.034s including waiting). Image size: 4269694 bytes.
Normal Created 71s kubelet Created container fix-permissions
Normal Started 71s kubelet Started container fix-permissions
Normal Pulled 67s kubelet Successfully pulled image "apache/nifi:latest" in 2.964s (2.964s including waiting). Image size: 1575137977 bytes.
Normal Pulling 65s (x2 over 70s) kubelet Pulling image "apache/nifi:latest"
Normal Created 62s (x2 over 67s) kubelet Created container nifi
Normal Pulled 62s kubelet Successfully pulled image "apache/nifi:latest" in 3.065s (3.065s including waiting). Image size: 1575137977 bytes.
Normal Started 61s (x2 over 67s) kubelet Started container nifi
Warning BackOff 60s kubelet Back-off restarting failed container nifi in pod nifi-deployment-67f49c6cd5-ntc48_default(bddadf64-e128-4afa-8f7c-3148b3a18c15)

kubectl logs

kubectl logs nifi-deployment-67f49c6cd5-ntc48
Defaulted container "nifi" out of: nifi, copy-default-config (init), override-nifi-properties (init), update-bootstrap-conf (init), fix-permissions (init)
File [/opt/nifi/nifi-current/conf/nifi.properties] uncommenting [nifi.python.command]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.python.extensions.source.directory.default]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.nar.library.autoload.directory]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.web.https.port]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.web.https.host]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.web.proxy.host]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.remote.input.host]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.remote.input.socket.port]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.remote.input.secure]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.protocol.is.secure]
File [/home/nifi/.nifi-cli.nifi.properties] replacing [baseUrl]
File [/home/nifi/.nifi-cli.nifi.properties] replacing [keystore]
File [/home/nifi/.nifi-cli.nifi.properties] replacing [keystoreType]
File [/home/nifi/.nifi-cli.nifi.properties] replacing [truststore]
File [/home/nifi/.nifi-cli.nifi.properties] replacing [truststoreType]
NIFI_WEB_PROXY_HOST was not set but NiFi is configured to run in a secure mode. The NiFi UI may be inaccessible if using port mapping or connecting through a proxy.
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.is.node]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.node.address]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.node.protocol.port]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.node.protocol.max.threads]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.load.balance.host]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.zookeeper.connect.string]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.zookeeper.root.node]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.flow.election.max.wait.time]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.flow.election.max.candidates]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.web.proxy.context.path]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.leader.election.implementation]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.state.management.provider.cluster]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.predict.enabled]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.predict.interval]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.query.interval]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.connection.model.implementation]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.connection.model.score.name]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.connection.model.score.threshold]
File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.leader.election.kubernetes.lease.prefix]
NiFi running with PID 45.
tail: cannot open '/opt/nifi/nifi-current/logs/nifi-app.log' for reading: No such file or directory

JAVA_HOME=/usr/lib/jvm/jdk-21.0.6-bellsoft-x86_64
NIFI_HOME=/opt/nifi/nifi-current

could you please look into it and revert what it's possible reason the container went in crashloopback?

Re: Apache Nifi CrashLoopBackOff on single node k8 cluster on docker desktop

VidyaSargur — Tue, 11 Feb 2025 09:59:34 GMT

@KRISHANKUMAR, Welcome to our community! To help you get the best possible answer, I have tagged our NiFi experts @satz @MattWho @SAMSAL who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

question Re: Apache Nifi CrashLoopBackOff on single node k8 cluster on docker desktop in Support Questions

Apache Nifi CrashLoopBackOff on single node k8 cluster on docker desktop

Re: Apache Nifi CrashLoopBackOff on single node k8 cluster on docker desktop