Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!
arshad10244
user rank
Explorer
My Accepted Solutions
Show All

ElasticSearchClientServiceImpl 1.22.0 x Elastic Se...

by Explorer in Support Questions
‎10-06-2023 11:56 PM
‎10-06-2023 11:56 PM
Hi Everyone, I'm having an issue while connecting to ES 7.17 with ElasticSearchClientServiceImpl 1.22.0 which seems to be a bug in the Rest Client. This is my configuration :   This is the error : This is what pops up in the bulletin board : I have verified connection to ES via postman and it works fine &  i have single node for ES and Nifi is also in a single node mode Any suggestions? Thanks ... View more
Labels:

Re: Convert Json array into csv using Nifi produce...

by Explorer in Support Questions
‎01-29-2023 08:49 AM
‎01-29-2023 08:49 AM
I had a similar issue, it happens because of nifi not being able to resolve the schema. You need to first create a schema and set the type of billing address as string and add it to avroschemaregistry and lastly configure your jsontreereader to use that schema ... View more

Re: Nifi JsonRecordSetWriter 1.17.0 Corrupting Dat...

by Explorer in Support Questions
‎01-29-2023 08:40 AM
‎01-29-2023 08:40 AM
Ok, i figured it out. For anyone having a similar issue: This happens when you are using a JSON array using any of the built-in readers, with the settings of schema set to "infer schema" and the data type within your array changes, For instance consider the following array : { "items": [ { "name": "John Doe", "type": [] }, { "name": "Jane Doe", "type": [ { "some_key": [ { "some_nested_key": "value" } ] } ] } ] } Notice in the items array, the type of first index is an empty array, in the second index the type has a nested array.   When using 'infer schema', nifi scans the first element to construct the schema on the fly, which means it would map the "type" key as an array, and store it without storing any schema for the nested array that's present in the second index, the result would be type:[MapRecord...]. The solution is to create your schema with entire structure first, simply remove the first element where type is an empty array and use http://www.dataedu.ca/avro to generate your schema, once that is done use AvroSchemaRegistry to save the schema and configure your recordreader to use the schema instead ... View more

Nifi JsonRecordSetWriter 1.17.0 Corrupting Data

by Explorer in Support Questions
‎01-29-2023 07:25 AM
‎01-29-2023 07:25 AM
Hello guys, I'm struggling with JsonRecordSetWriter 1.17.0, we have valid nested JSON data, but for some reason whenever this data passes through JsonRecordSetWriter, on nested arrays it adds the following :     "orderItemsCustomizes" : [ "MapRecord[{quantity=1, itemCurrency=AED, totalPrice=0, orderItemId=2767, itemCustomizeGroup=MapRecord[{itemModifierGroupAttributes=[Ljava.lang.Object;@4658e00e, customizeGroupName=Burger Selection}], itemCustomize=MapRecord[{itemCustomizeGroupId=2308, itemId=18622, itemName=Chicken Burger, isAvailabel=true, updated_at=2023-01-27 15:59:31, itemCurrency=AED, itemCost=0.00, created_at=2023-01-27 15:59:31, itemModifierAttributes=[Ljava.lang.Object;@ee87ab1, id=7539, itemImage=https://d2cvcbugmdflrn.cloudfront.net/e777bb72-a445-4162-9844-f9bda5e7a3d0.jpg}], itemId=18622, itemCustomizeGroupId=2308, createdAt=2023-01-27 16:48:02, itemCustomizeId=7539, price=0.00, ID=1153, updatedAt=2023-01-27 16:48:02}]", "MapRecord[{quantity=1, itemCurrency=AED, totalPrice=2, orderItemId=2767, itemCustomizeGroup=MapRecord[{itemModifierGroupAttributes=[Ljava.lang.Object;@d33701d, customizeGroupName=Fries Selection}], itemCustomize=MapRecord[{itemCustomizeGroupId=2309, itemId=18622, itemName=French Fries, isAvailabel=true, updated_at=2023-01-27 15:59:31, itemCurrency=AED, itemCost=2.00, created_at=2023-01-27 15:59:31, itemModifierAttributes=[Ljava.lang.Object;@f5ca635, id=7542, itemImage=null}], itemId=18622, itemCustomizeGroupId=2309, createdAt=2023-01-27 16:48:02, itemCustomizeId=7542, price=2.00, ID=1154, updatedAt=2023-01-27 16:48:02}]" ],     I've seen a bunch of posts where this is happening, any idea why does this happen?   ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎10-07-2023 04:40 AM
Community Statistics
Member Since ‎01-29-2023 07:14 AM
Last Visited ‎10-07-2023 04:40 AM
Posts 4