Member since
07-29-2020
306
Posts
89
Kudos Received
89
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
57 | 09-16-2023 07:20 AM | |
82 | 09-12-2023 08:55 AM | |
110 | 09-08-2023 06:35 AM | |
104 | 08-31-2023 03:26 PM | |
173 | 08-31-2023 09:15 AM |
09-18-2023
07:17 AM
1 Kudo
Hi @manishg , This has been asked before in a different way but you can implement the same method: https://community.cloudera.com/t5/Support-Questions/Wait-for-a-Flowfile-to-be-picked-only-after-the-previous/m-p/375890#M242696 If that helps please accept solution. Thanks
... View more
09-16-2023
07:20 AM
Hi @mr80132 , Something I noticed about your configuration for the "GenerateTableFetch" processor is that you are not setting any value for the "Maximum-value Columns". I think you need to set at least one column name that the processor will track the max value for and fetch anything that comes with value greater than the max. Please refer to the processor description: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.12.1/org.apache.nifi.processors.standard.GenerateTableFetch/ if that helps please accept solution . Thanks
... View more
09-13-2023
06:19 AM
I dont think you can use PutSQL for this. Try ExecuteSQL or you might have to use ExecuteScript processor as other posts suggest: https://community.cloudera.com/t5/Support-Questions/Does-ExecuteSQL-processor-allow-to-execute-stored-procedure/m-p/158922 https://stackoverflow.com/questions/72348844/get-output-of-stored-procedure-using-putsql-in-nifi If that helps please accept solution. Thanks
... View more
09-12-2023
08:55 AM
1 Kudo
Hi @CommanderLaus , The easiest way I can think of is to use the ExtractText Processor where you define an attribute as dynamic property and assign the Regex as follows: I did only the first three fields but you can do the others the same way. If that helps please accept solution. Thanks
... View more
09-11-2023
10:56 AM
Looking at the UnpackContent processor code , it seems to be writing the items into memory. https://github.com/kiranjilla/nifi-xom/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java#L202 The absolute.path appears to be getting set there. Not sure if the proposed solution will work. @MattWho , @cotopaul , @steven-matison can you guys help with this?
... View more
09-08-2023
12:43 PM
Hi @MmSs , Can you provide more details about your flow? For example what do you do after the UnpackContent processor and what processor\flow are you using to copy the file to the final location? For example if you are using PutFile after the UnpackContent where different files with the same filename from different packages are getting overwritten, while at the sametime you have another process\flow like GetFile\ListFile\FetchFile to copy the unzipped files to the final target then you are copying files that are constantly being verwritten (through UnpackContnet->PutFile) , this of course will make the files corrupt. The solution would be is when saving the file after the UnpackContent make sure to save it under unique name or path to prevent the conflict. You can use some of the UnpackContent processor write attributes and other upstream processors write attributes to help comeup with a unique name for the unzipped files ( segment.original.filename , path, filename, fragment.identifier , fragment.index ...etc.) If you find this is helpful please accept solution. Thanks
... View more
09-08-2023
06:35 AM
1 Kudo
Hi @manishg , Only the top processor in a given flow can be set to execute on "Primary Node". No downstream processor can be set as such. If you want downstream processors to execute on a single node , then open the Queue configuration going to the FetchXMLFile , Under Settings tab , select "Single node" from the Load Balance Strategy" drop down. However selecting this option there is no guarantees which node is it going to be as the manual indicates: " Single node: All FlowFiles will be sent to a single node in the cluster. Which node they are sent to is not configurable. If the node is disconnected from the cluster or if unable to communicate with the node, the data that is queued for that node will remain queued until the node is available again. " If that helps please accept solution. Thanks
... View more
08-31-2023
03:26 PM
1 Kudo
Hi @RRG , The below example can give you an idea of what processors you might need to use to achieve what you are looking for. You might adjust processors as needed. Feel free to let me know if you have any questions. If that helps please accept solution. Thanks
... View more
08-31-2023
09:15 AM
Another way to consider for processing one file at a time: https://www.youtube.com/watch?v=kvJx8vQnCNE&lc=UgzHBAI-4X3vn_dWSHV4AaABAg.9ssoo7vzq1d9ssqH-c7ttL
... View more
08-31-2023
09:12 AM
If you are getting multiple records in one Json array , then you probably need to use SplitJosn Processor to get each record individually , then extract the values you need using EvaluateJsonPath from each record then do the PutSQL.
... View more