About arshad10244

arshad10244 · ‎10-06-2023

Hi Everyone, I'm having an issue while connecting to ES 7.17 with ElasticSearchClientServiceImpl 1.22.0 which seems to be a bug in the Rest Client. This is my configuration : This is the error : This is what pops up in the bulletin board : I have verified connection to ES via postman and it works fine & i have single node for ES and Nifi is also in a single node mode Any suggestions? Thanks

arshad10244 · ‎01-29-2023

I had a similar issue, it happens because of nifi not being able to resolve the schema. You need to first create a schema and set the type of billing address as string and add it to avroschemaregistry and lastly configure your jsontreereader to use that schema

arshad10244 · ‎01-29-2023

Ok, i figured it out. For anyone having a similar issue: This happens when you are using a JSON array using any of the built-in readers, with the settings of schema set to "infer schema" and the data type within your array changes, For instance consider the following array : { "items": [ { "name": "John Doe", "type": [] }, { "name": "Jane Doe", "type": [ { "some_key": [ { "some_nested_key": "value" } ] } ] } ] } Notice in the items array, the type of first index is an empty array, in the second index the type has a nested array. When using 'infer schema', nifi scans the first element to construct the schema on the fly, which means it would map the "type" key as an array, and store it without storing any schema for the nested array that's present in the second index, the result would be type:[MapRecord...]. The solution is to create your schema with entire structure first, simply remove the first element where type is an empty array and use http://www.dataedu.ca/avro to generate your schema, once that is done use AvroSchemaRegistry to save the schema and configure your recordreader to use the schema instead

arshad10244 · ‎01-29-2023

Hello guys, I'm struggling with JsonRecordSetWriter 1.17.0, we have valid nested JSON data, but for some reason whenever this data passes through JsonRecordSetWriter, on nested arrays it adds the following : "orderItemsCustomizes" : [ "MapRecord[{quantity=1, itemCurrency=AED, totalPrice=0, orderItemId=2767, itemCustomizeGroup=MapRecord[{itemModifierGroupAttributes=[Ljava.lang.Object;@4658e00e, customizeGroupName=Burger Selection}], itemCustomize=MapRecord[{itemCustomizeGroupId=2308, itemId=18622, itemName=Chicken Burger, isAvailabel=true, updated_at=2023-01-27 15:59:31, itemCurrency=AED, itemCost=0.00, created_at=2023-01-27 15:59:31, itemModifierAttributes=[Ljava.lang.Object;@ee87ab1, id=7539, itemImage=https://d2cvcbugmdflrn.cloudfront.net/e777bb72-a445-4162-9844-f9bda5e7a3d0.jpg}], itemId=18622, itemCustomizeGroupId=2308, createdAt=2023-01-27 16:48:02, itemCustomizeId=7539, price=0.00, ID=1153, updatedAt=2023-01-27 16:48:02}]", "MapRecord[{quantity=1, itemCurrency=AED, totalPrice=2, orderItemId=2767, itemCustomizeGroup=MapRecord[{itemModifierGroupAttributes=[Ljava.lang.Object;@d33701d, customizeGroupName=Fries Selection}], itemCustomize=MapRecord[{itemCustomizeGroupId=2309, itemId=18622, itemName=French Fries, isAvailabel=true, updated_at=2023-01-27 15:59:31, itemCurrency=AED, itemCost=2.00, created_at=2023-01-27 15:59:31, itemModifierAttributes=[Ljava.lang.Object;@f5ca635, id=7542, itemImage=null}], itemId=18622, itemCustomizeGroupId=2309, createdAt=2023-01-27 16:48:02, itemCustomizeId=7542, price=2.00, ID=1154, updatedAt=2023-01-27 16:48:02}]" ], I've seen a bunch of posts where this is happening, any idea why does this happen?

Online	Offline
Last Visited	‎12-20-2023 05:46 AM

Member Since	‎01-29-2023 07:14 AM
Last Visited	‎12-20-2023 05:46 AM
Posts	4
Kudos received	3

Cloudera Community

Re: Nifi JsonRecordSetWriter 1.17.0 Corrupting Dat...

ElasticSearchClientServiceImpl 1.22.0 x Elastic Se...

Re: Convert Json array into csv using Nifi produce...

Re: Nifi JsonRecordSetWriter 1.17.0 Corrupting Dat...

Nifi JsonRecordSetWriter 1.17.0 Corrupting Data