Member since
07-29-2020
439
Posts
173
Kudos Received
133
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
125 | 03-20-2024 02:58 AM | |
231 | 03-20-2024 02:07 AM | |
266 | 03-15-2024 06:10 AM | |
227 | 03-11-2024 07:33 PM | |
374 | 03-08-2024 04:17 AM |
03-15-2023
08:36 AM
1 Kudo
Hi, First of all make sure that the provided json is in valid format . What you provided in your example doesnt have double quotes surrounding the key\value as in { "Label1": "Value1", "Label2": "Value2" } Assuming thejson is valid , you can use the following jolt spec to produce expected output: [
{
"operation": "shift",
"spec": {
"*": "GeneralLabel[#].&"
}
}
] To learn more about jolt transformation please refer to : https://jolt-demo.appspot.com/#inception https://intercom.help/godigibee/en/articles/4044359-transformer-getting-to-know-jolt If that helps please accept solution. Thanks
... View more
03-13-2023
10:43 AM
Hi @lben , It seems what you need is the Wait\Notify processors . For more information on how to use those processors : https://pierrevillard.com/2018/06/27/nifi-workflow-monitoring-wait-notify-pattern-with-split-and-merge/ https://www.youtube.com/watch?v=ALvzZ6D4GtA let me know how that works for you. If that helps please accept solution. Thanks
... View more
03-09-2023
08:11 AM
Hi @rob1 , Please try the following transformation: [
// first shift will group all strategies under one collection: saleStrategyCollection
{
"operation": "shift",
"spec": {
"saleStrategyCollection": "&",
"saleStrategy": "saleStrategyCollection[].&"
}
},
// 2ed shift will bucket each strategy & code as an array item ([&2]=0,1,2..)
// under saleStrategies collection and use the same key (&)
{
"operation": "shift",
"spec": {
"saleStrategyCollection": {
"*": { // level 2 = 0,1,2..
"saleStrategy": { //level 1
"strategy": "saleStrategies[&2].&", //level 0
"code": "saleStrategies[&2].&"
}
}
}
}
}
] If that helps, please accept solution. Thanks
... View more
03-05-2023
04:40 PM
I apologize, I forgot that you are talking about an Avro format and not json. In this I think you need to pass your own avro schema. You can first convert Avro to Json using ConvertAvroToJSON processor , then use ConvertRecrod which can be configured as follows: In the AvroRecordWrite , you can set your schema with the proper record\namespace values. For Example in my case I used the "User Schema Text" strategy and either provide the schema directly in the Schema Text Property or reference it using flowfile attribute ${avro.schema} using Expression Language. The Schema Text : {
"type": "record",
"name": "bank",
"namespace": "org.xyz.com",
"fields": [
{
"name": "AGENT_CODE",
"type": [
"null",
"string"
]
},
{
"name": "CORR_ACC_NO",
"type": [
"null",
"string"
]
},
{
"name": "LOCAL_ACC_NO",
"type": [
"null",
"string"
]
}
]
} Hope that helps
... View more
03-05-2023
06:39 AM
Hi, You have few options like UpdateRecord Processor or JoltJsonTrnasformation. I recommend the UpdateRecord , for more information on how to use please refer to : https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.12.1/org.apache.nifi.processors.standard.UpdateRecord/ https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.12.1/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html It should be straight forward if you use the "Replace with Literal" strategy. If that helps please accept solution Thanks "
... View more
03-04-2023
09:24 AM
Hi, Please try the following jolt spec: [
{
"operation": "shift",
"spec": {
"rows": {
"*": {
"f": {
"0": {
"v": "[#4].export_time"
},
"1": {
"v": "[#4].account_id"
},
"2": {
"v": "[#4].cost"
}
}
}
}
}
}
] If that helps please accept solution. Thanks
... View more
02-27-2023
06:29 AM
1 Kudo
Hi, I dont think the EvaluateJsonPath and RouteOnAttribute will work properly since you have an array of json records in the the incoming flowfiles. You have to split the array using SplitJson processor and then use the EvaluateJsonPath->RouteOnAttributes to get the expected result. However there is better and more efficient option than using the mentioned processors (Split->Evaluate->Route), you can use just QueryRecords to filter the array and isolate the needed records. You can see example on how to use QueryRecords here: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.19.1/org.apache.nifi.processors.standard.QueryRecord/additionalDetails.html If that helps , please accept solution.
... View more
02-19-2023
08:16 AM
2 Kudos
Hi, Try to look into QueryRecord or PartitionRecord Processors. Those might help. Thanks
... View more
02-17-2023
07:08 AM
Hi, thanks for the information , I came across situation before and I'm not sure if there is better way but you can use the following processors after you split to data into individual records : ExtractText -> PUTSQL For the ExtractText: 1- Add dynamic Property to capture all the Json Content from incoming flowfile as follows: Note: You have to be careful with if each record data can be large (> 1024 Chars). In this case you need to look into modifying "Maximum Buffer Size" and "Maximum Capture Group Length" accordingly otherwise the data will be truncated. For the PutSQL: Once you configure the JDBC Connection Pool , you can set the SQL Statement Property with something like this: insert into myTable (jsonCol) values ('${JsonRecord}') If that helps please accept solution
... View more
02-16-2023
09:36 AM
Hi , Can you provide sample\example data of the json and how do you expect to save it as ?
... View more