Member since
07-29-2020
574
Posts
323
Kudos Received
176
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2155 | 12-20-2024 05:49 AM | |
| 2452 | 12-19-2024 08:33 PM | |
| 2201 | 12-19-2024 06:48 AM | |
| 1463 | 12-17-2024 12:56 PM | |
| 2107 | 12-16-2024 04:38 AM |
04-15-2023
08:23 AM
Hi, If you are looking to set the id value for the first element in the columns array you can use the following spec: [
{
"operation": "shift",
"spec": {
"columns": {
"0": {
"value": "[].@(1,name)"
}
}
}
}
] If that helps please accept solution.
... View more
04-12-2023
08:24 AM
1 Kudo
hi, You can take a look into ForkEnrichment and JoinEnrichment processors : https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.20.0/org.apache.nifi.processors.standard.JoinEnrichment/additionalDetails.html If that helps, please accept solution. Thanks
... View more
03-26-2023
06:10 AM
Hi, I noticed that you have an "Original" relationship going from SplitJson back to itself. This where your duplicates are coming from. The "Original" relationship means the original input flowfile that is passed to the splitJson from JoltTransformation processor. You need to terminate this relationship and not pass it to anything unless you are using Wait-Notify processors which I dont think in your case. You can terminate relationship by going to splitJson Configuration then go to the Relationship tab and check the terminate box under the Original relationship.
... View more
03-23-2023
05:58 AM
What do you mean by it gave you "so many flowfiles". If you follow the same jolt spec and split processors configuration as I provided it should give you the exact amount flowfiles similar to the number of ACT in the original input. Can you provide more details?
... View more
03-19-2023
08:13 AM
3 Kudos
Hi , My guess is that you are trying to use ConvertJsonToSQL & PutSQL to transpose the json into sql table. For this to happen you cant use the input json in the given format , you need to transform it using JoltTransformJSON processor to create an array of records where each record consist of "sbrActivityType" & "sbrDateTime" columns. The json keys have to match the table column names for this to work. The jolt spec to use can look like this: [
{
"operation": "shift",
"spec": {
"sbrActivityType": {
// convert every activity into an array element
"*": "data[#1].sbrActivityType"
},
"sbrDateTime": "sbrDateTime"
}
},
{
"operation": "modify-default-beta",
"spec": {
"data": {
"*": {
// create new sbrDateTime key under each array element from
// above and set its value to the initial sbrDateTime
"sbrDateTime": "@(3,sbrDateTime)"
}
}
}
}
] you set the Jolt Specification property with the spec above in the JoltTransformJSON This should give you new json flowfile as follows: {
"data" : [ {
"sbrActivityType" : "ACT+A",
"sbrDateTime" : "Sun Mar 19 13:45:15 AST 2023"
}, {
"sbrActivityType" : "ACT+X",
"sbrDateTime" : "Sun Mar 19 13:45:15 AST 2023"
}, {
"sbrActivityType" : "ACT+A",
"sbrDateTime" : "Sun Mar 19 13:45:15 AST 2023"
}, {
"sbrActivityType" : "ACT+X",
"sbrDateTime" : "Sun Mar 19 13:45:15 AST 2023"
}, {
"sbrActivityType" : "ACT+X",
"sbrDateTime" : "Sun Mar 19 13:45:15 AST 2023"
} ],
"sbrDateTime" : "Sun Mar 19 13:45:15 AST 2023"
} After that, use SplitJosn to split each record into its own json flowfile. The split JsonPath Expression is set to $.data.*: Then you use ConvertJSONToSQL as follows: And Finally you use the PUTSQL processor as follows: This should populate the table accordingly: If this helps please accept solution. Thanks
... View more
03-15-2023
08:36 AM
1 Kudo
Hi, First of all make sure that the provided json is in valid format . What you provided in your example doesnt have double quotes surrounding the key\value as in { "Label1": "Value1", "Label2": "Value2" } Assuming thejson is valid , you can use the following jolt spec to produce expected output: [
{
"operation": "shift",
"spec": {
"*": "GeneralLabel[#].&"
}
}
] To learn more about jolt transformation please refer to : https://jolt-demo.appspot.com/#inception https://intercom.help/godigibee/en/articles/4044359-transformer-getting-to-know-jolt If that helps please accept solution. Thanks
... View more
03-13-2023
10:43 AM
Hi @lben , It seems what you need is the Wait\Notify processors . For more information on how to use those processors : https://pierrevillard.com/2018/06/27/nifi-workflow-monitoring-wait-notify-pattern-with-split-and-merge/ https://www.youtube.com/watch?v=ALvzZ6D4GtA let me know how that works for you. If that helps please accept solution. Thanks
... View more
03-09-2023
08:11 AM
Hi @rob1 , Please try the following transformation: [
// first shift will group all strategies under one collection: saleStrategyCollection
{
"operation": "shift",
"spec": {
"saleStrategyCollection": "&",
"saleStrategy": "saleStrategyCollection[].&"
}
},
// 2ed shift will bucket each strategy & code as an array item ([&2]=0,1,2..)
// under saleStrategies collection and use the same key (&)
{
"operation": "shift",
"spec": {
"saleStrategyCollection": {
"*": { // level 2 = 0,1,2..
"saleStrategy": { //level 1
"strategy": "saleStrategies[&2].&", //level 0
"code": "saleStrategies[&2].&"
}
}
}
}
}
] If that helps, please accept solution. Thanks
... View more
03-05-2023
04:40 PM
I apologize, I forgot that you are talking about an Avro format and not json. In this I think you need to pass your own avro schema. You can first convert Avro to Json using ConvertAvroToJSON processor , then use ConvertRecrod which can be configured as follows: In the AvroRecordWrite , you can set your schema with the proper record\namespace values. For Example in my case I used the "User Schema Text" strategy and either provide the schema directly in the Schema Text Property or reference it using flowfile attribute ${avro.schema} using Expression Language. The Schema Text : {
"type": "record",
"name": "bank",
"namespace": "org.xyz.com",
"fields": [
{
"name": "AGENT_CODE",
"type": [
"null",
"string"
]
},
{
"name": "CORR_ACC_NO",
"type": [
"null",
"string"
]
},
{
"name": "LOCAL_ACC_NO",
"type": [
"null",
"string"
]
}
]
} Hope that helps
... View more
03-05-2023
06:39 AM
Hi, You have few options like UpdateRecord Processor or JoltJsonTrnasformation. I recommend the UpdateRecord , for more information on how to use please refer to : https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.12.1/org.apache.nifi.processors.standard.UpdateRecord/ https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.12.1/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html It should be straight forward if you use the "Replace with Literal" strategy. If that helps please accept solution Thanks "
... View more