Member since
08-18-2019
56
Posts
11
Kudos Received
18
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1187 | 11-15-2023 05:38 AM | |
| 4543 | 07-12-2023 05:38 AM | |
| 1086 | 01-10-2023 01:50 AM | |
| 1489 | 12-06-2022 12:08 AM | |
| 5523 | 12-02-2022 12:55 AM |
09-26-2022
07:08 AM
Hello, you can configure that just 1 Flowfile will be handled inside of a specific ProcessorGroup. There is the config option on PG which is called: Process Group FlowFile Concurrency for that you can set the value: Single FlowFile per Node After PutKudu you will destroy the FlowFile or route it out from ProcessorGroup, then the next FlowFile will be released to enter to ProcessorGroup In your case the Flow would look like: ListFile Processor -> ProcessorGroup (handels fetchFile, data Transformation and putKudu)
... View more
02-18-2022
12:33 AM
what if the value is empty. How to set null value instead of empty string ?
... View more
01-21-2021
07:06 AM
@Lallagreta Make sure you do not have any line returns in the values for your dynamic properties added in the UpdateAttribute processor. When you click on the value field for each property you should not see a line "2". For example: Above would result in the value assigned to the FlowFile Attribute having a line return. If this is the case, edit the properties value(s) to remove the line returns so you only see one line (1). Hope this helps, Matt
... View more
01-13-2021
07:51 AM
@FaerballertThanks again. It turned out that I just should ignore the question marks in the SQL statement. Everything works for now.
... View more
01-13-2021
04:47 AM
Did you already think about to transform your csv to json and then rebuild it with JOLT?
... View more
01-08-2021
05:23 AM
2 Kudos
@murali2425 The solution you are looking for is QueryRecord configured with a CSV Record Reader and Record Writer. You also have UpdateRecord and ConvertRecord which can use the Readers/Writers. This method is preferred over splitting the file and adds some nice functionality. This method allows you to provide a schema for both the inbound csv (reader) and the downstream csv (writer). Using QueryRecord you should be able to split the file, and set attribute of filename set to column1. At the end of the flow you should be able to leverage that filename attribute to resave the new file. You can find some specific examples and configuration screen shots here: https://community.cloudera.com/t5/Community-Articles/Running-SQL-on-FlowFiles-using-QueryRecord-Processor-Apache/ta-p/246671 If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post. Thanks, Steven
... View more
01-07-2021
04:23 AM
Hi Faerballert Thanks for helping . I am able to get entire content into attribute . I have one doubt that i have attribute name called "JSONValue" so that extracted value should be in this name but the flowfile content saves into the attribute "JSONValue.0 " as supposed to create value in JSONValue attribute . Please see screen-print for the same . I had to enable "Include Capture Group 0" as True otherwise the Regex not accepting and expecting value 0 to40 other than false.
... View more
01-03-2021
06:33 AM
Sorry, cant visualize it, but your flow can be look like this: QueryDatabaseTableRecord -> Wait ->PutS3Object
-> FetchFile -> EvaluateJsonPath -> Notify You can overhand the attribute from JSON to the waiting FlowFile. If you need help with configuration or other stuff for this, just text me.
... View more
12-04-2020
07:04 AM
try this "createdon" : {
"$date" : "putInHereDate"
} instead of "createdon": ISODate("2017-03-03") you can solve this also via JOLT
... View more
11-20-2020
12:03 AM
my csv is file.csv ok scheme dont need. is possible auto-create table with column names from first line csv?
... View more