Member since
09-26-2024
12
Posts
6
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
536 | 10-22-2024 10:16 AM |
12-02-2024
12:47 AM
1 Kudo
hi How do we get a report for all the file transfers for a given period ? We are using Nifi to only transfer file from one location to another using SFTP and no transformation on file data. Would like to generate a report with the transferred file name and its transfer status. Thanks
... View more
Labels:
- Labels:
-
Apache NiFi
11-14-2024
04:09 AM
1 Kudo
Hi I'm new to NiFi and exploring my options. We are planning to use NiFi as a File Movement platform, where we will be transferring thousands of files everyday of sizes ranging upto 50GB without any extraction, transformation or loading. We just need to pull the file from one location and push to another location using SFTP and/or S3. Is NiFi a right tool to use to do this activity ? Thanks
... View more
Labels:
- Labels:
-
Apache NiFi
11-04-2024
12:21 AM
1 Kudo
@joseomjr Thanks for the reply. So you are reading the file from local using Groovy processor which is invoked in InvokeScriptedProcessor ??
... View more
11-01-2024
08:23 PM
1 Kudo
@MattWho Thanks for you response. Use case is basically to only upload file using API, and not to download. We have an UI which is used upload a file, On click of choose file we were planning to call our backend API which inturn calls NiFi HandleHTTPRequest API. Authorization will part of UI and API. Also, NiFi API will not be directly exposed externally. Since we were exploring our options in Nifi, we are trying this approach. If you feel this is not achievable or not the right approach, we can drop this and look into other options. Thank you
... View more
11-01-2024
09:43 AM
1 Kudo
Hi How do I host an API using HandleHttpRequest which can be used to upload files to NiFi server from local using "multipart/form-data". Have tried below approach, however I get read permission issue as the Nifi user can't access local data. Please suggest an approach which can be used to upload file from local. Thanks
... View more
Labels:
- Labels:
-
Apache NiFi
10-22-2024
10:40 AM
Thanks Matt, Was able to resolve the issue with your putFile solution.
... View more
10-22-2024
10:16 AM
Hi Matt, Closing this query as it's not related to S3 issue. Thanks for your response.
... View more
10-21-2024
11:32 AM
For the record Able to resolve the fetching issue, there was port number missing in the overridden URL. However, now i get a different error when writing the file to local disk. PutFile[id=07413ab5-d3d2-1e9a-99b0-c0f57682f17c] Penalizing FlowFile[filename=20242323/year/year.txt] and transferring to failure: org.apache.nifi.processor.exception.FlowFileAccessException: Failed to export StandardFlowFileRecord[uuid=35019b7d-1e33-44db-8432-6a54b2f6586e,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1729535188764-436, container=default, section=436], offset=70, length=16],offset=0,name=20242323/year/year.txt,size=16] to /apps/fex/shared/mina/archive/.20242323/year/year.txt due to java.io.FileNotFoundException: /apps/fex/shared/mina/archive/.20242323/year/year.txt (No such file or directory) - Caused by: java.io.FileNotFoundException: /apps/fex/shared/mina/archive/.20242323/year/year.txt (No such file or directory)
... View more
10-21-2024
10:12 AM
Credentials and region are correct as I'm able to fetch the file if they are directly under home directory (20242323) of the bucket. The issue is fetching files that are under folder/subfolders like "month" or "year" in this case. ListS3 Properties FetchS3Object Properties
... View more