Member since
02-10-2025
1
Post
0
Kudos Received
0
Solutions
02-10-2025
09:58 PM
i am facing challenge to deploy the apache nifi on single node k8 cluster in docker desktop. Here is my nifi-config.yml and nifi-deployment.yml file with following error message. nifi-config.yml apiVersion: v1 kind: ConfigMap metadata: name: nifi-config data: nifi.properties: | nifi.web.http.host=0.0.0.0 nifi.web.http.port=8080 # Site to Site properties (if needed) # nifi.remote.input.host=... # nifi.remote.input.secure=... # nifi.remote.input.socket.port=... # nifi.remote.input.http.enabled=... # nifi.remote.input.http.transaction.ttl=30 sec # HTTPS properties (if applicable) # nifi.web.https.host=... # nifi.web.https.port=... # nifi.web.https.network.interface.default=... # nifi.web.https.application.protocols=h2 http/1.1 # TLS Cipher Suites (if applicable) # nifi.web.https.ciphersuites.include=... # nifi.web.https.ciphersuites.exclude=... nifi-deployment.yml apiVersion: apps/v1 kind: Deployment metadata: name: nifi-deployment labels: app: nifi spec: replicas: 1 selector: matchLabels: app: nifi template: metadata: labels: app: nifi spec: # Init Container 1: Copy the entire default configuration directory from the image initContainers: - name: copy-default-config image: apache/nifi:latest command: - sh - -c - | cp -R /opt/nifi/nifi-current/conf/* /writable/ volumeMounts: - name: writable-config mountPath: /writable # Init Container 2: Override the default nifi.properties with our custom version - name: override-nifi-properties image: busybox:latest command: - sh - -c - | cp /config/nifi.properties /writable/nifi.properties volumeMounts: - name: config-volume mountPath: /config - name: writable-config mountPath: /writable # Init Container 3: Update bootstrap.conf to force foreground mode - name: update-bootstrap-conf image: busybox:latest command: - sh - -c - | if [ -f /writable/bootstrap.conf ]; then # Replace the line even if commented (handles both commented and uncommented lines) sed -i 's/^#\?nifi\.bootstrap\.runInForeground=.*/nifi.bootstrap.runInForeground=true/' /writable/bootstrap.conf else echo "bootstrap.conf not found in /writable!" exit 1 fi volumeMounts: - name: writable-config mountPath: /writable # Init Container 4: Fix permissions on the entire configuration directory so that files are owned by UID 1000 (the nifi user) - name: fix-permissions image: busybox:latest command: - sh - -c - | chown -R 1000:1000 /writable volumeMounts: - name: writable-config mountPath: /writable containers: - name: nifi image: apache/nifi:latest ports: - containerPort: 8080 env: - name: NIFI_WEB_HTTP_PORT value: "8080" # Mount the complete configuration directory over NiFi’s conf directory volumeMounts: - name: writable-config mountPath: /opt/nifi/nifi-current/conf volumes: - name: writable-config emptyDir: {} - name: config-volume configMap: name: nifi-config kubectl describe pod kubectl describe pod nifi-deployment-67f49c6cd5-ntc48 Name: nifi-deployment-67f49c6cd5-ntc48 Namespace: default Priority: 0 Service Account: default Node: docker-desktop/192.168.65.3 Start Time: Tue, 11 Feb 2025 11:23:07 +0530 Labels: app=nifi pod-template-hash=67f49c6cd5 Annotations: <none> Status: Running IP: 10.1.1.102 IPs: IP: 10.1.1.102 Controlled By: ReplicaSet/nifi-deployment-67f49c6cd5 Init Containers: copy-default-config: Container ID: docker://72e35257e520eaa5f37a7c2e0583cdd5f8991d35617f6d4407ae600d9293238c Image: apache/nifi:latest Image ID: docker-pullable://apache/nifi@sha256:6f8de66ca2f8eae07588508be7c40559077f646abbeb7472de7d855e38817b52 Port: <none> Host Port: <none> Command: sh -c cp -R /opt/nifi/nifi-current/conf/* /writable/ State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 11 Feb 2025 11:23:12 +0530 Finished: Tue, 11 Feb 2025 11:23:13 +0530 Ready: True Restart Count: 0 Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zh25w (ro) /writable from writable-config (rw) override-nifi-properties: Container ID: docker://1a4ba0873de81c5e87616ed19b1da1119be5bb4d6c16dbb772efa156721a888a Image: busybox:latest Image ID: docker-pullable://busybox@sha256:a5d0ce49aa801d475da48f8cb163c354ab95cab073cd3c138bd458fc8257fbf1 Port: <none> Host Port: <none> Command: sh -c cp /config/nifi.properties /writable/nifi.properties State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 11 Feb 2025 11:23:16 +0530 Finished: Tue, 11 Feb 2025 11:23:16 +0530 Ready: True Restart Count: 0 Environment: <none> Mounts: /config from config-volume (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zh25w (ro) /writable from writable-config (rw) update-bootstrap-conf: Container ID: docker://b92faf93707dafa2b9a1e8de95ac097227453128064da3cb7d71cfb513966a18 Image: busybox:latest Image ID: docker-pullable://busybox@sha256:a5d0ce49aa801d475da48f8cb163c354ab95cab073cd3c138bd458fc8257fbf1 Port: <none> Host Port: <none> Command: sh -c if [ -f /writable/bootstrap.conf ]; then # Replace the line even if commented (handles both commented and uncommented lines) sed -i 's/^#\?nifi\.bootstrap\.runInForeground=.*/nifi.bootstrap.runInForeground=true/' /writable/bootstrap.conf else echo "bootstrap.conf not found in /writable!" exit 1 fi State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 11 Feb 2025 11:23:21 +0530 Finished: Tue, 11 Feb 2025 11:23:21 +0530 Ready: True Restart Count: 0 Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zh25w (ro) /writable from writable-config (rw) fix-permissions: Container ID: docker://c4ad5c183ec1d92609ac104c41419f7206da9a8258d72af9235ed8efc757a133 Image: busybox:latest Image ID: docker-pullable://busybox@sha256:a5d0ce49aa801d475da48f8cb163c354ab95cab073cd3c138bd458fc8257fbf1 Port: <none> Host Port: <none> Command: sh -c chown -R 1000:1000 /writable State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 11 Feb 2025 11:23:24 +0530 Finished: Tue, 11 Feb 2025 11:23:24 +0530 Ready: True Restart Count: 0 Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zh25w (ro) /writable from writable-config (rw) Containers: nifi: Container ID: docker://8e137edcf6f9be8e54b96930db92d6c3385829ac4a6508eed80a8f8fde90f7b3 Image: apache/nifi:latest Image ID: docker-pullable://apache/nifi@sha256:6f8de66ca2f8eae07588508be7c40559077f646abbeb7472de7d855e38817b52 Port: 8080/TCP Host Port: 0/TCP State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 11 Feb 2025 11:24:26 +0530 Finished: Tue, 11 Feb 2025 11:24:27 +0530 Last State: Terminated Reason: Completed Exit Code: 0 Started: Tue, 11 Feb 2025 11:23:50 +0530 Finished: Tue, 11 Feb 2025 11:23:52 +0530 Ready: False Restart Count: 3 Environment: NIFI_WEB_HTTP_PORT: 8080 Mounts: /opt/nifi/nifi-current/conf from writable-config (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-zh25w (ro) Conditions: Type Status PodReadyToStartContainers True Initialized True Ready False ContainersReady False PodScheduled True Volumes: writable-config: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: <unset> config-volume: Type: ConfigMap (a volume populated by a ConfigMap) Name: nifi-config Optional: false kube-api-access-zh25w: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 87s default-scheduler Successfully assigned default/nifi-deployment-67f49c6cd5-ntc48 to docker-desktop Normal Pulling 87s kubelet Pulling image "apache/nifi:latest" Normal Pulled 83s kubelet Successfully pulled image "apache/nifi:latest" in 4.098s (4.098s including waiting). Image size: 1575137977 bytes. Normal Created 83s kubelet Created container copy-default-config Normal Started 82s kubelet Started container copy-default-config Normal Pulling 82s kubelet Pulling image "busybox:latest" Normal Pulled 79s kubelet Successfully pulled image "busybox:latest" in 2.942s (2.942s including waiting). Image size: 4269694 bytes. Normal Created 79s kubelet Created container override-nifi-properties Normal Started 79s kubelet Started container override-nifi-properties Normal Pulling 78s kubelet Pulling image "busybox:latest" Normal Pulled 74s kubelet Successfully pulled image "busybox:latest" in 3.368s (3.368s including waiting). Image size: 4269694 bytes. Normal Created 74s kubelet Created container update-bootstrap-conf Normal Started 74s kubelet Started container update-bootstrap-conf Normal Pulling 74s kubelet Pulling image "busybox:latest" Normal Pulled 71s kubelet Successfully pulled image "busybox:latest" in 3.034s (3.034s including waiting). Image size: 4269694 bytes. Normal Created 71s kubelet Created container fix-permissions Normal Started 71s kubelet Started container fix-permissions Normal Pulled 67s kubelet Successfully pulled image "apache/nifi:latest" in 2.964s (2.964s including waiting). Image size: 1575137977 bytes. Normal Pulling 65s (x2 over 70s) kubelet Pulling image "apache/nifi:latest" Normal Created 62s (x2 over 67s) kubelet Created container nifi Normal Pulled 62s kubelet Successfully pulled image "apache/nifi:latest" in 3.065s (3.065s including waiting). Image size: 1575137977 bytes. Normal Started 61s (x2 over 67s) kubelet Started container nifi Warning BackOff 60s kubelet Back-off restarting failed container nifi in pod nifi-deployment-67f49c6cd5-ntc48_default(bddadf64-e128-4afa-8f7c-3148b3a18c15) kubectl logs kubectl logs nifi-deployment-67f49c6cd5-ntc48 Defaulted container "nifi" out of: nifi, copy-default-config (init), override-nifi-properties (init), update-bootstrap-conf (init), fix-permissions (init) File [/opt/nifi/nifi-current/conf/nifi.properties] uncommenting [nifi.python.command] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.python.extensions.source.directory.default] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.nar.library.autoload.directory] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.web.https.port] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.web.https.host] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.web.proxy.host] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.remote.input.host] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.remote.input.socket.port] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.remote.input.secure] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.protocol.is.secure] File [/home/nifi/.nifi-cli.nifi.properties] replacing [baseUrl] File [/home/nifi/.nifi-cli.nifi.properties] replacing [keystore] File [/home/nifi/.nifi-cli.nifi.properties] replacing [keystoreType] File [/home/nifi/.nifi-cli.nifi.properties] replacing [truststore] File [/home/nifi/.nifi-cli.nifi.properties] replacing [truststoreType] NIFI_WEB_PROXY_HOST was not set but NiFi is configured to run in a secure mode. The NiFi UI may be inaccessible if using port mapping or connecting through a proxy. File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.is.node] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.node.address] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.node.protocol.port] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.node.protocol.max.threads] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.load.balance.host] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.zookeeper.connect.string] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.zookeeper.root.node] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.flow.election.max.wait.time] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.flow.election.max.candidates] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.web.proxy.context.path] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.leader.election.implementation] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.state.management.provider.cluster] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.predict.enabled] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.predict.interval] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.query.interval] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.connection.model.implementation] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.connection.model.score.name] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.analytics.connection.model.score.threshold] File [/opt/nifi/nifi-current/conf/nifi.properties] replacing [nifi.cluster.leader.election.kubernetes.lease.prefix] NiFi running with PID 45. tail: cannot open '/opt/nifi/nifi-current/logs/nifi-app.log' for reading: No such file or directory JAVA_HOME=/usr/lib/jvm/jdk-21.0.6-bellsoft-x86_64 NIFI_HOME=/opt/nifi/nifi-current could you please look into it and revert what it's possible reason the container went in crashloopback?
... View more
Labels:
- Labels:
-
Apache NiFi