Skip to main content
rupeshh
rupeshh
Silver ⭐️⭐️⭐️
June 12, 2023
Solved

How to use docker bind mount directory in executestreamcommand processor nifi

Hello everyone, I am new to docker and I have been researching and working with nifi recently. I have one query for my use case where I need to use code repositories that are outside the docker container (I have already bind them while docker run command) and I can access the folder via exec but I need to access these scripts in the executestreamcommand processor's working directory and command path fields. Even after putting the destination directories into the fields I am getting no such file or directory exists. So, please help me out.

steven-matison
steven-matisonAnswer
Platinum ⭐️⭐️
June 14, 2023

@rupeshh Docker container are never fun for this and other reasons.   I still think you have missing permissions on the file.  I cannot see the ownership of the file listing, and i cannot see the path in the error.  At any rate, the error definitely suggests the processor does not see the file.     

One suggestion would be to use the nifi user and cli on docker to ls the directory and files.  If that user cannot see the files,  that would indicate the same issue the error states (directory path or file does not exist, or not seen due to permissions).

 

View in thread

8 replies

DianaTorres
Community Manager
DianaTorres
Community Manager
June 12, 2023

@rupeshh Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi expert @steven-matison   who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Translate
Regards, Diana Torres, Senior Community Moderator
rupeshh
rupeshhAuthor
Silver ⭐️⭐️⭐️
June 13, 2023

Thanks! @DianaTorres . I will be waiting for @steven-matison to response.

Translate
steven-matison
Platinum ⭐️⭐️
steven-matison
Platinum ⭐️⭐️
June 13, 2023

@rupeshh In order for nifi to be able to see the mounted directory or files within it, it needs to be properly owned to the same user that is running nifi.  For example:

 

chown nifi:nifi /some/path   

 Then nifi will be able to see the directories and files.

Translate
rupeshh
rupeshhAuthor
Silver ⭐️⭐️⭐️
June 14, 2023

Hello @steven-matison thanks for replying but I am getting permission denied everytime I am using chmod or chown in the docker shell and sudo does not work in the shell so I don't know how exactly I should change the ownership. My directory is /tmp/nifi/src/respository_1 and /tmp/nifi/src/respository_2 and repository_1 and repository_2 are already owned by nifi user but the /tmp/nifi/src all these 3 owned by root and I think I will have to change the ownership of each directory because in the docker run command I have mounted to the /tmp/nifi/src/repository_1 and /tmp/nifi/src/repository_2. Can you please help me out?

Translate
steven-matison
Platinum ⭐️⭐️
steven-matisonAnswer
Platinum ⭐️⭐️
June 14, 2023

@rupeshh Docker container are never fun for this and other reasons.   I still think you have missing permissions on the file.  I cannot see the ownership of the file listing, and i cannot see the path in the error.  At any rate, the error definitely suggests the processor does not see the file.     

One suggestion would be to use the nifi user and cli on docker to ls the directory and files.  If that user cannot see the files,  that would indicate the same issue the error states (directory path or file does not exist, or not seen due to permissions).

 

Translate
    rupeshh
    rupeshhAuthor
    Silver ⭐️⭐️⭐️
    June 16, 2023

    Thanks @steven-matison I found out it was permission issue actually. I am able to access the directory now. A big thanks!

    Translate
      Prabash
      Prabash
      Silver ⭐️
      September 13, 2023

      Hi @rupeshh I am also facing a similar situation and wanted to clarify from you a few things. I see in the Configuration for the ExecuteStreamCommand you have referred to the venv inside the repository for the command path, does that mean you have the venv within the docker container? or are you referring to the one outside the docker container (in the host machine)?

       
       

       

       

      Translate
      Terms & ConditionsAccessibility statement