Member since
07-29-2020
574
Posts
323
Kudos Received
176
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 3742 | 12-20-2024 05:49 AM | |
| 3998 | 12-19-2024 08:33 PM | |
| 3788 | 12-19-2024 06:48 AM | |
| 2448 | 12-17-2024 12:56 PM | |
| 3206 | 12-16-2024 04:38 AM |
09-16-2023
07:20 AM
Hi @mr80132 , Something I noticed about your configuration for the "GenerateTableFetch" processor is that you are not setting any value for the "Maximum-value Columns". I think you need to set at least one column name that the processor will track the max value for and fetch anything that comes with value greater than the max. Please refer to the processor description: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.12.1/org.apache.nifi.processors.standard.GenerateTableFetch/ if that helps please accept solution . Thanks
... View more
09-12-2023
08:55 AM
1 Kudo
Hi @CommanderLaus , The easiest way I can think of is to use the ExtractText Processor where you define an attribute as dynamic property and assign the Regex as follows: I did only the first three fields but you can do the others the same way. If that helps please accept solution. Thanks
... View more
09-11-2023
10:56 AM
Looking at the UnpackContent processor code , it seems to be writing the items into memory. https://github.com/kiranjilla/nifi-xom/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java#L202 The absolute.path appears to be getting set there. Not sure if the proposed solution will work. @MattWho , @cotopaul , @steven-matison can you guys help with this?
... View more
09-08-2023
12:43 PM
Hi @MmSs , Can you provide more details about your flow? For example what do you do after the UnpackContent processor and what processor\flow are you using to copy the file to the final location? For example if you are using PutFile after the UnpackContent where different files with the same filename from different packages are getting overwritten, while at the sametime you have another process\flow like GetFile\ListFile\FetchFile to copy the unzipped files to the final target then you are copying files that are constantly being verwritten (through UnpackContnet->PutFile) , this of course will make the files corrupt. The solution would be is when saving the file after the UnpackContent make sure to save it under unique name or path to prevent the conflict. You can use some of the UnpackContent processor write attributes and other upstream processors write attributes to help comeup with a unique name for the unzipped files (segment.original.filename, path, filename,fragment.identifier, fragment.index ...etc.) If you find this is helpful please accept solution. Thanks
... View more
09-08-2023
06:35 AM
1 Kudo
Hi @manishg , Only the top processor in a given flow can be set to execute on "Primary Node". No downstream processor can be set as such. If you want downstream processors to execute on a single node , then open the Queue configuration going to the FetchXMLFile , Under Settings tab , select "Single node" from the Load Balance Strategy" drop down. However selecting this option there is no guarantees which node is it going to be as the manual indicates: " Single node: All FlowFiles will be sent to a single node in the cluster. Which node they are sent to is not configurable. If the node is disconnected from the cluster or if unable to communicate with the node, the data that is queued for that node will remain queued until the node is available again. " If that helps please accept solution. Thanks
... View more
08-31-2023
03:26 PM
1 Kudo
Hi @RRG , The below example can give you an idea of what processors you might need to use to achieve what you are looking for. You might adjust processors as needed. Feel free to let me know if you have any questions. If that helps please accept solution. Thanks
... View more
08-31-2023
09:15 AM
Another way to consider for processing one file at a time: https://www.youtube.com/watch?v=kvJx8vQnCNE&lc=UgzHBAI-4X3vn_dWSHV4AaABAg.9ssoo7vzq1d9ssqH-c7ttL
... View more
08-31-2023
09:12 AM
If you are getting multiple records in one Json array , then you probably need to use SplitJosn Processor to get each record individually , then extract the values you need using EvaluateJsonPath from each record then do the PutSQL.
... View more
08-29-2023
07:20 PM
If you are able to store the display name and the mail values into flowfile attributes, then the easiest way is to use the PutSQL processor to create your insert statement of those values and other values you want to ingest into the target table. In the PutSQL you can specify the insert statement in the SQL Statement property as follows: insert into myTable(displayNameCol, mailCol, otherCol...) values
('${displayNameAttribute}','${mailAttribute}', 'otherValue',...) The AttributeToJson should work as well if you specify which attributes you want to insert into the DB to be converted as Json flowfile and then use the PutDatabaseRecord with JsonTreeReader where you specify that the Statement Type property as INSERT. The only caveat here is in the PutDatabaseRecord the attribute name should match the table column names. If that helps please accept solution. Thanks
... View more
08-29-2023
09:19 AM
Hi @Kiranq , If you are getting the output form the ExecuteStreamCommand as Json I dont understand why are you using the ConvertAvroJson Processor. In the PutDatabaseRecrod you can specify the Record Reader property as JsonTreeReader and consume it as such. I guess if you can shed more light on how the executestream output looks like maybe we can help you better. If that helps please accept solution. Thanks
... View more