Member since
02-01-2022
288
Posts
103
Kudos Received
60
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1265 | 05-15-2025 05:45 AM | |
| 5297 | 06-12-2024 06:43 AM | |
| 8296 | 04-12-2024 06:05 AM | |
| 6137 | 12-07-2023 04:50 AM | |
| 3411 | 12-05-2023 06:22 AM |
08-08-2023
08:58 AM
1 Kudo
@JohnSilver Open the queue list between list/fetch and look at the content of flowfile (0.00 size). Take the (i) or the (👁) in the list to go deeper. Also inspect the attributes tab attached to the flowfile. Looking for file names, path, etc... those bits need to be parsed to fetchSMB as attributes, and then the right file will be fetched for putHDFS.
... View more
08-08-2023
04:56 AM
1 Kudo
@JohnSilver A few advices: In the future post screen shot of flow, and important processor configuration screen shots. You can also share a flow definition file on github, and sample data so we can help see what you see. Without those some advice is below. It sounds like ListSMB is returning a list of all results( files and or paths). Can you share that output? (Run ListSMB once, inspect flowfile, share here in a code block). Without seeing that I think another processor should iterate through that list flowfile contents, and fetch each file/path independently.
... View more
08-07-2023
07:09 AM
@ibrooks Excellent article here. I will definitely be using this in my CML Demos going foward. I just sent a PR w/ some flow improvements you may like to use as well.
... View more
08-07-2023
06:51 AM
2 Kudos
@sam_s0ni I took up this conversation internally and exposed another related JIRA 11763. This behavior was removed in 1.19 and will not be coming back due to nature of parameter evaluation against references of those parameters when they are changed. You can view the conversation for NIFI-11763 here: https://issues.apache.org/jira/browse/NIFI-11763
... View more
08-07-2023
06:40 AM
@Sivaluxan Assuming Huawei OBS is s3 compliant( a quick search says it is), you should be able to use all of the S3/OBS based nifi processors. You would just need to satisfy the access/auth requirements in order to connect from NiFi to H OBS from within the of the processors below. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.22.0/org.apache.nifi.processors.aws.s3.ListS3/index.html https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.22.0/org.apache.nifi.processors.aws.s3.FetchS3Object/index.html https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.11.4/org.apache.nifi.processors.aws.s3.PutS3Object/index.html
... View more
07-17-2023
08:43 AM
@Aimen I would be willing to help you get started. Shoot me a DM to co-ordinate a working session.
... View more
07-17-2023
08:38 AM
@Madhav_VD This is a job that nifi can do itself. You would just tail the log file w/ nifi TailFile. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.20.0/org.apache.nifi.processors.standard.TailFile/ Once you have the log file going into flowfiles in nifi, you can build your flow to filter lines you want to versus lines you do not want. Then when ready use any flavor of database processor (ExecuteSQLRecord, ExecuteSQL, PutSQL, PutDatabase, etc) to translate the log line into a sql insert statement.
... View more
07-17-2023
08:32 AM
@Aimen We will need to see screen shots of your flow, and the configuration for the s3 processor. It also helps to share the exact error, etc. Without this information, it is not possible to suggest solutions.
... View more
07-12-2023
08:44 AM
1 Kudo
@rupeshh , make a new post and describe your solution, then click Accept As Solution!! Then this post will be available in future for anyone experiencing similar. Great job getting it rectified!
... View more
07-10-2023
05:32 AM
@cotopaul This may be worthy of a JIRA and a fix. PutBigQueryBatch is going away, so if there is some function missing in PutBigQuery we need to get it in there. I just did some work on PutBigQuery, let me have a discussion with @davidh and I will get back to this asap. If you can, share an example flow.
... View more