- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
sftp donot fetch files
- Labels:
-
Apache NiFi
Created on
01-25-2025
06:16 AM
- last edited on
01-26-2025
08:50 PM
by
VidyaSargur
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello , i am trying to use sftp with nifi .
this is my docker image :
volumes:
nifi-conf:
services:
nifi:
image: apache/nifi:latest
container_name: nifi
ports:
- "8089:8443"
- "5656:5656"
volumes:
- nifi-conf:/opt/nifi/nifi-current/conf
environment:
NIFI_WEB_PROXY_HOST: localhost:8089
SINGLE_USER_CREDENTIALS_USERNAME: admin
SINGLE_USER_CREDENTIALS_PASSWORD:
sftp:
image: atmoz/sftp
volumes:
- ./sftp/upload:/home/foo/
ports:
- "2222:22"
command: foo:pass:1001
at first the found the sftplist running with no output and when i stop it i get 2 active threads and canot stop or start it just terminal
when the docker image load and debug it i founds that the owner of /home/foo not foo it is root and when i change it manually i get bad ownership error of chroot directory
Created 01-26-2025 08:52 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@ose_gold, welcome to our community! To help you get the best possible answer, I have tagged our NiFi experts, @MattWho, @SAMSAL, and @Shelton, who may be able to assist you further.
Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.
Regards,
Vidya Sargur,Community Manager
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
Created 01-27-2025 01:18 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The SFTP issues appear to stem from incorrect permissions and ownership in your Docker setup. Here's the analysis and solution:
Key Issues:
- Root ownership instead of user 'foo' for /home/foo
- Incorrect chroot directory permissions
- Docker volume mount permissions
You have a volume nifi-conf:/opt/nifi/nifi-current/conf for the NiFi container, but it's not declared in the volumes section at the bottom check the addition below
Docker Compose has newer versions 3.8 It might be a good idea to update to a more recent version depending on the features you need.
version: '3' #Docker Compose has newer versions 3.8
services:
nifi:
image: apache/nifi:latest # Consider specifying a version
container_name: nifi
ports:
- "8089:8443"
- "5656:5656"
volumes:
- nifi-conf:/opt/nifi/nifi-current/conf
environment:
NIFI_WEB_PROXY_HOST: localhost:8089
SINGLE_USER_CREDENTIALS_USERNAME: admin
SINGLE_USER_CREDENTIALS_PASSWORD: {xxyourpasswdxx}
sftp:
image: atmoz/sftp
volumes:
- ./sftp/upload:/home/foo/upload
ports:
- "2222:22"
command: foo:pass:1001
# Add these permissions
user: "1001:1001"
environment:
- CHOWN_USERS=foo
- CHOWN_DIRS=/home/foo
volumes:
nifi-conf:
Before starting containers
# Set correct permissions on host
mkdir -p ./sftp/upload
chown -R 1001:1001 ./sftp/upload
chmod 755 ./sftp/upload
This configuration:
- Sets proper user/group ownership
- Maintains correct chroot permissions
- Ensures volume mount permissions are preserved
- Prevents permission conflicts between host and container
Happy Hadooping
