Support Questions
Find answers, ask questions, and share your expertise

referencing attributes in Nifi Record after JSON nested object is converted to Avro schema record

New Contributor

Hi Guys,
I am doing some nifi extraction and are trying to work out how to do something with a nested json array

I have a json array like this
{"device":"JPE19151536","interface":"Port-Channel3.2512","key":"Port-Channel3.2512","rates":{"inBitsRate":{"value":4.687787409031554},"inPktsRate":{"value":0.007918559808362655},"outBitsRate":{"value":1.1345431741251526E-119},"outPktsRate":{"value":3.972490105480225E-123},"statsUpdateTime":1.602638448008944E9},"statistics":{"inOctets":4297765,"outOctets":336472,"inErrors":0,"outErrors":0,"inDiscards":0,"outDiscards":0,"inTotalPkts":0,"inUcastPkts":51714,"outUcastPkts":1085,"inBroadcastPkts":0,"inMulticastPkts":0,"outBroadcastPkts":0,"outMulticastPkts":0,"lastUpdate":1.602638448008944E9}}

I then converted the data to an avro record with a schema ready to write to influx after schema is applied the data looks like this in the flow file with the formatted option applied. excuse the different record data in in example from the first step

[ {
"device" : "JPE19151536",
"interface" : "Ethernet12.1202",
"key" : "Ethernet12.1202",
"rates" : {
"inBitsRate" : {
"value" : 0
},
"inPktsRate" : {
"value" : 0
},
"outBitsRate" : {
"value" : 0
},
"outPktsRate" : {
"value" : 0
},
"statsUpdateTime" : 1.60263846E9
},
"statistics" : {
"inOctets" : 0,
"outOctets" : 0,
"inErrors" : 0,
"outErrors" : 0,
"inDiscards" : 0,
"outDiscards" : 0,
"inTotalPkts" : 0,
"inUcastPkts" : 0,
"outUcastPkts" : 0,
"inBroadcastPkts" : 0,
"inMulticastPkts" : 0,
"outBroadcastPkts" : 0,
"outMulticastPkts" : 0,
"lastUpdate" : 1.60263846E9
}
} ]

I then want to write the statistics.inOctets and many others as fields in the PutInfluxDatabaseRecord process if I add device,interface,key as tags in the Tags property then tags get written to influx correctly. But when it comes to fields if I try to use anything from statistics or rates I get item not present in record. Not sure if I am referencing the fields incorrectly or its an issue referencing a sub object of the data I have tried statistics.inOctets $.statistics.inOctets

Avro Schema looks like this
{
"name": "AristaPerfInterface",
"type": "record",
"namespace": "com.acme.avro",
"fields": [
{
"name": "device",
"type": "string"
},
{
"name": "interface",
"type": "string"
},
{
"name": "key",
"type": "string"
},
{
"name": "rates",
"type": {
"name": "rates",
"type": "record",
"fields": [
{
"name": "inBitsRate",
"type": {
"name": "inBitsRate",
"type": "record",
"fields": [
{
"name": "value",
"type": "int"
}
]
}
},
{
"name": "inPktsRate",
"type": {
"name": "inPktsRate",
"type": "record",
"fields": [
{
"name": "value",
"type": "int"
}
]
}
},
{
"name": "outBitsRate",
"type": {
"name": "outBitsRate",
"type": "record",
"fields": [
{
"name": "value",
"type": "int"
}
]
}
},
{
"name": "outPktsRate",
"type": {
"name": "outPktsRate",
"type": "record",
"fields": [
{
"name": "value",
"type": "int"
}
]
}
},
{
"name": "statsUpdateTime",
"type": "float"
}
]
}
},
{
"name": "statistics",
"type": {
"name": "statistics",
"type": "record",
"fields": [
{
"name": "inOctets",
"type": "int"
},
{
"name": "outOctets",
"type": "int"
},
{
"name": "inErrors",
"type": "int"
},
{
"name": "outErrors",
"type": "int"
},
{
"name": "inDiscards",
"type": "int"
},
{
"name": "outDiscards",
"type": "int"
},
{
"name": "inTotalPkts",
"type": "int"
},
{
"name": "inUcastPkts",
"type": "int"
},
{
"name": "outUcastPkts",
"type": "int"
},
{
"name": "inBroadcastPkts",
"type": "int"
},
{
"name": "inMulticastPkts",
"type": "int"
},
{
"name": "outBroadcastPkts",
"type": "int"
},
{
"name": "outMulticastPkts",
"type": "int"
},
{
"name": "lastUpdate",
"type": "float"
}
]
}
}
]
}

Would appreciate some advice thanks
Phil

0 REPLIES 0