Member since
10-11-2022
3
Posts
0
Kudos Received
0
Solutions
10-15-2022
07:09 PM
Hi Green Really appreciate the assistance, if anything its probably me not clearly articulating what I'm trying to achieve, two weeks in with nifi The raw content from the API is received as follows "{ "ret_code" : 0, "ret_msg" : "OK", "ext_code" : "", "ext_info" : "", "result" : [ { "id" : "187715622692", "symbol" : "BTCUSDT", "price" : 19109.5, "qty" : 0.004, "side" : "Buy", "time" : "2022-10-16T01:25:31.000Z", "trade_time_ms" : 1665883531832, "is_block_trade" : false }, { "id" : "187715618142", "symbol" : "BTCUSDT", "price" : 19109.5, "qty" : 0.882, "side" : "Buy", "time" : "2022-10-16T01:25:31.000Z", "trade_time_ms" : 1665883531123, "is_block_trade" : false }, { "id" : "187715614682", "symbol" : "BTCUSDT", "price" : 19109.5, "qty" : 0.001, "side" : "Buy", "time" : "2022-10-16T01:25:30.000Z", "trade_time_ms" : 1665883530414, "is_block_trade" : false" Likely I was trying to over engineer with the jolt, and ended up with { "id" : [ 1.84310970522E11, 1.84310967802E11 ], "symbol" : [ "BTCUSDT", "BTCUSDT" ], "price" : [ 19241.5, 19241.0 ], "qty" : [ 1.896, 0.002 ], "side" : [ "Buy", "Sell" ], "time" : [ "2022-10-11T12:26:21.000Z", "2022-10-11T12:26:21.000Z" ], "trade_time_ms" : [ 1.665491181666E12, 1.665491181604E12 ], "is_block_trade" : [ false, false ] } Where if I just used a json split, which gave the following { "id" : "187715622692", "symbol" : "BTCUSDT", "price" : 19109.5, "qty" : 0.004, "side" : "Buy", "time" : "2022-10-16T01:25:31.000Z", "trade_time_ms" : 1665883531832, "is_block_trade" : false } The format above was accepted by PutDatabaseRecord, where the issue moved to duplicate ID being written, setting ID as primary key seems have stopped the duplicate being written but is probably not the cleanest solution... The other API Im dealing with is where I think the solution you suggested with be the ideal use case... { "lastUpdateId" : 579582125552, "E" : 1665885750096, "T" : 1665885750088, "symbol" : "BTCUSD_PERP", "pair" : "BTCUSD", "bids" : [ [ "19139.5", "8824" ], [ "19139.4", "757" ] "asks" : [ [ "19139.6", "3165" ], [ "19139.7", "812" ] } Where the requirement would be to extract for example "bids" : [ [ "19139.5", "8824" ], [ "19139.4", "757" ] into two seperate record .. maintaining the fields in each
... View more