Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

sftp donot fetch files

avatar
Visitor

Hello , i am trying to use sftp with nifi .

this is my docker image : 

volumes:
nifi-conf:

services:
nifi:
image: apache/nifi:latest
container_name: nifi
ports:
- "8089:8443"
- "5656:5656"
volumes:
- nifi-conf:/opt/nifi/nifi-current/conf
environment:
NIFI_WEB_PROXY_HOST: localhost:8089
SINGLE_USER_CREDENTIALS_USERNAME: admin
SINGLE_USER_CREDENTIALS_PASSWORD: 

sftp:
image: atmoz/sftp
volumes:
- ./sftp/upload:/home/foo/
ports:
- "2222:22"
command: foo:pass:1001

at first the found the sftplist running with no output and when i stop it i get 2 active threads and canot stop or start it just terminal  

when the docker image load and debug it i founds that the owner of /home/foo not foo it is root and when i change it manually i get bad ownership error of chroot directory  

ose_gold_0-1737814389746.png

2 REPLIES 2

avatar
Community Manager

@ose_gold, welcome to our community! To help you get the best possible answer, I have tagged our NiFi experts, @MattWho, @SAMSAL, and @Shelton, who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Master Mentor

@ose_gold 

The SFTP issues appear to stem from incorrect permissions and ownership in your Docker setup. Here's the analysis and solution:

Key Issues:

  1. Root ownership instead of user 'foo' for /home/foo
  2. Incorrect chroot directory permissions
  3. Docker volume mount permissions

You have a volume nifi-conf:/opt/nifi/nifi-current/conf for the NiFi container, but it's not declared in the volumes section at the bottom check the addition below
Docker Compose has newer versions 3.8 It might be a good idea to update to a more recent version depending on the features you need.

 

version: '3'  #Docker Compose has newer versions 3.8
services:
  nifi:
    image: apache/nifi:latest   # Consider specifying a version
    container_name: nifi
    ports:
      - "8089:8443"
      - "5656:5656"
    volumes:
      - nifi-conf:/opt/nifi/nifi-current/conf
    environment:
      NIFI_WEB_PROXY_HOST: localhost:8089
      SINGLE_USER_CREDENTIALS_USERNAME: admin
      SINGLE_USER_CREDENTIALS_PASSWORD: {xxyourpasswdxx}

  sftp:
    image: atmoz/sftp
    volumes:
      - ./sftp/upload:/home/foo/upload
    ports:
      - "2222:22"
    command: foo:pass:1001
    # Add these permissions
    user: "1001:1001"
    environment:
      - CHOWN_USERS=foo
      - CHOWN_DIRS=/home/foo
   volumes:
     nifi-conf:

 

Before starting containers

 

# Set correct permissions on host
mkdir -p ./sftp/upload
chown -R 1001:1001 ./sftp/upload
chmod 755 ./sftp/upload

 

This configuration:

  • Sets proper user/group ownership
  • Maintains correct chroot permissions
  • Ensures volume mount permissions are preserved
  • Prevents permission conflicts between host and container

Happy Hadooping