Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Need help on JOLT spec

Highlighted

Need help on JOLT spec

New Contributor

HI All,

Is there an easy way to capture object index ?
INPUT:

{ "header": { "namespace": "someNamespace", "version": "1" }, "body": { "dataM": { "sourceM": [ { "name": "State", "type": "string" }, { "name": "City", "type": "string" }, { "name": "var", "type": "string" }, { "name": "unit", "type": "string" }, { "name": "type", "type": "string" } ], "valueM": [ { "name": "timestamp", "type": "date-time" }, { "name": "value", "type": "string" }, { "name": "quality", "type": "integer" }, { "name": "error", "type": "string" } ] }, "data": [ { "source": [ "Austria", "Vienna", "temp", "F", "now" ], "values": [ [ "2018-10-02T15:00:00.05Z", "3.1234", 7, "SomeError" ], [ "2018-10-02T15:00:46.05Z", "3.221", 0, "SomeOtherError" ] ] }, { "source": [ "Germany", "Berlin", "pressure", "PA", "now" ], "values": [ [ "2018-10-02T16:00:00.05Z", "5", 0, "NoError" ] ] } ] } }


spec:

[{ "operation": "shift", "spec": { "body": { "data": { "*": { "@(4,header)": { "*": "newdata[&2].&" }, "source": { "*": "newdata.[&2].@(4,dataM.sourceM.[&].name)" }, "values": { "*": { "*": "newdata.[&3].@(5,dataM.valueM[&].name)" } } } } } } } , { "operation": "shift", "spec": { "newdata": { "*": "[&0]" } } } ]


Desired Output:

[ { "namespace" : "someNamespace", "version" : "1", "State" : "Austria", "City" : "Vienna", "var" : "temp", "unit" : "F", "type" : "now", "timestamp" :"2018-10-02T15:00:00.05Z", "value" : "3.1234", "quality" : 7, "error" : "SomeError" }, { "namespace" : "someNamespace", "version" : "1", "State" : "Austria", "City" : "Vienna", "var" : "temp", "unit" : "F", "type" : "now", "timestamp" : "2018-10-02T15:00:46.05Z" , "value" : "3.221" , "quality" : 0, "error" : "" },{ "namespace" : "someNamespace", "version" : "1", "State" : "Germany", "City" : "Berlin", "var" : "pressure", "unit" : "PA", "type" : "now", "timestamp" : "2018-10-02T16:00:00.05Z", "value" : "5", "quality" : 0, "error" : "NoError" } ]


Currently, I'm getting arrays for timestamp, value, quality and error for each object. Instead, I need source array repeated per number of values within the data object.