Member since
02-11-2021
29
Posts
1
Kudos Received
0
Solutions
08-29-2023
12:36 AM
Having the same issue while loading data from NIFI 1.21 to Kudu. Anyone find the solution?. I can connect to Kudu Via Hue and Impala
... View more
05-08-2023
09:20 AM
i have slightly modifed the spec as. The matchingStatus_value doesn't provide the right result [{ "operation": "shift", "spec": { "header": { "timeStamp": "records.inv_activity_ts", "activityId": "records.inv_activity_id", "action": "records.action" }, "resource": { "drniId": "records.inv_id", "subtype": "records.inv_subtype", "name": "records.inv_name", "resourceCharacteristic": { "*": { "name": { "tmf639_dataSource_ingestion_uuid": { "@(2,value)": "records.tmf639_dataSource_ingestion_uuid" }, "createdDate": { "@(2,value)": "records.createdDate" }, "lastModifiedDate": { "@(2,value)": "records.lastModifiedDate" }, "status": { "@(2,value)": "records.status" } } } } } } }] Is this a valid one?
... View more
05-08-2023
06:40 AM
For the below given input, how do i move the name value pairs(installDate and status) to the upper level { "header": { "version": "1.0", "timeStamp": "1670484663189", "activityId": "256388257993155783", "action": "create", "refreshId": "null" }, "resource": { "drniId": "256383859946641699", "@type": "LogicalCrossConnection", "subtype": "Backplane Connection", "name": "Backplane Connection", "resourceCharacteristic": [ { "name": "tmf639_dataSource_ingestion_uuid", "value": "256383859946641699", "@type": "string" }, { "name": "p16Latest", "value": true, "@type": "boolean" }, { "name": "installDate", "value": "202303323", "@type": "string" }, { "name": "status", "value": "In Service", "@type": "string" } ], "resourceRelationship": [ { "drniId": "256387639517862117", "@type": "LogicalInterface:HAS_CONNECTION_COMPONENT:OUT", "subtype": "ODU2e", "name": "ODU2e.1-8", "relationDrniId": "256383859946641700", "resourceRelationshipCharacteristic": [ { "name": "p16Latest", "value": true, "@type": "boolean" }, { "name": "sectionNumber", "value": 0, "@type": "long" }, { "name": "p3CreateActivity", "value": 256388257993155780, "@type": "long" }, { "name": "isNotInRoute", "value": "N", "@type": "string" }, { "name": "p2Create", "value": 1670480203578, "@type": "long" }, { "name": "seq", "value": 1, "@type": "long" } ] }, { "drniId": "256383859946641335", "@type": "LogicalInterface:HAS_CONNECTION_COMPONENT:OUT", "subtype": "ODU2e", "name": "ODU2e.1-8", "relationDrniId": "256383859946641701", "resourceRelationshipCharacteristic": [ { "name": "p16Latest", "value": true, "@type": "boolean" }, { "name": "sectionNumber", "value": 0, "@type": "long" }, { "name": "p3CreateActivity", "value": 256388257993155780, "@type": "long" }, { "name": "isNotInRoute", "value": "N", "@type": "string" }, { "name": "latest", "value": [ 2, 3, 16 ], "@type": "array" } ] } ] } } Expected Output { "records" : { "inv_activity_ts" : "1670484663189", "inv_activity_id" : "256388257993155783", "action" : "create", "inv_id" : "256383859946641699", "inv_subtype" : "Backplane Connection", "inv_name" : "Backplane Connection", "installDate" : "202303323", "status" : "status" } } @SAMSAL Any help you could provide?
... View more
Labels:
- Labels:
-
Apache NiFi
05-08-2023
05:29 AM
I have used the below spec for moving some of the resource characterstic to top level but iam getting null values. [ { "operation": "shift", "spec": { "header": { "timeStamp": "records.inv_activity_ts", "activityId": "records.inv_activity_id", "action": "records.action" }, "resource": { "drniId": "records.inv_id", "subtype": "records.inv_subtype", "name": "records.inv_name", "resourceCharacteristic": { "*": { "name": { "matchingkey": "records.matchingkey-value", "status": "records.status-value" }, "value": { "matchingkey": "records.matchingkey-value", "status": "records.status-value" } } } } } } ] Output iam getting { "records" : { "inv_activity_ts" : "1670484663189", "inv_activity_id" : "256388257993155783", "action" : "create", "inv_id" : "256383859946641699", "inv_subtype" : "Backplane Connection", "inv_name" : "Backplane Connection", "matchingkey-value" : null, "status-value" : null } } What is mistake iam making?.
... View more
05-02-2023
07:11 PM
Thanks @SAMSAL You are a genius. I was caught with the duplicate index array and couldn't resolve.
... View more
05-01-2023
05:08 PM
Basically inhertied properties of header and resource inside resourceRelationshipCharacteristic { "records" : [ { "relationDrniId" : "5143209344417655493", "timeStamp" : "1681441547144", "activityId" : "5142917973836294867", "action" : "modify", "name": "p3Latest", "value": true, "@type": "boolean" }, { "relationDrniId" : "5143209344417655493", "timeStamp" : "1681441547144", "activityId" : "5142917973836294867", "action" : "modify", "name": "isNotInRoute", "value": "N", "@type": "string" } , { "relationDrniId" : "5143209344417655494", "timeStamp" : "1681441547144", "activityId" : "5142917973836294867", "action" : "modify", "name": "p3Latest", "value": true, "@type": "boolean" } , "relationDrniId" : "5143209344417655494", "timeStamp" : "1681441547144", "activityId" : "5142917973836294867", "action" : "modify", "name": "isNotInRoute", "value": "N", "@type": "string" } ] }
... View more
04-30-2023
08:10 PM
Thanks @SAMSAL .It worked. In the same way how do i get an simillar output for "resourceRelationshipCharacteristic" . i tried few option but not getting the desirable output. { "header": { "version": "1.0", "timeStamp": "1681441547144", "activityId": "5142917973836294867", "action": "modify" }, "resource": { "drniId": "261968731599587823", "@type": "VVVVV", "subtype": "BBBBBB", "name": "VVVVVVV", "resourceCharacteristic": [ { "name": "matchingkey", "value": "CCCCC", "@type": "string" } ], "resourceRelationship": [ { "drniId": "261968731599587635", "@type": "CCCCCC", "subtype": "OTU4", "name": "BBBBBB", "relationDrniId": "5143209344417655493", "resourceRelationshipCharacteristic": [ { "name": "p3Latest", "value": true, "@type": "boolean" }, { "name": "isNotInRoute", "value": "N", "@type": "string" } ] }, { "drniId": "261988866406271344", "@type": "BVDFDFS", "subtype": "VDFDG", "name": "ABCC", "relationDrniId": "5143209344417655494", "resourceRelationshipCharacteristic": [ { "name": "p3Latest", "value": true, "@type": "boolean" }, { "name": "isNotInRoute", "value": "N", "@type": "string" } ] } ] } }
... View more
04-26-2023
08:17 AM
Hi Jolt Experts, Iam looking to use the Jolt transformation in NIFI and looking for a help. { "header": { "timeStamp": "1681441547144", "activityId": "5142917973836294867" }, "resource": { "drniId": "261968731599587823", "@type": "Order", "subtype": "Retail", "name": "CCCCC", "resourceCharacteristic": [ { "propname": "aaa", "value": "P_1655442" }, { "name": "latest", "value": [ 11, 200, 300, 16, 2, 3 ] }] } } To have an output like { "records" : [ { "timeStamp" : "1681441547144", "activityId" : "5142917973836294867", "drniId" : "261968731599587823", "type" : "Order", "subtype" : "Retail", "name" : "CCCCC", "propname" : "aaa", "value" : "P_1655442" }, { "timeStamp" : "1681441547144", "activityId" : "5142917973836294867", "drniId" : "261968731599587823", "type" : "Order", "subtype" : "Retail", "name" : "CCCCC", "propname" : "latest", "value" : "11,200,300,16,2,3" } ] } Basically copying all the top header and resource into to new by flattening it. Any help appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
12-05-2022
01:54 AM
Is there a way in NIFI to create Hive tables from using the schema from CDP schema registry.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache NiFi
09-22-2022
09:03 PM
While converting the input flow file JSON to any other format using query record (CSVwriter or AVRO Writer) using inferschema strategy the NIFI processor is trying to convert to Date based on first few characters of the incoming string. NIFI Error Output Successfully parsed a JSON object from input but failed to convert into a Record object with the given schema\n- Caused by: org.apache.nifi.serialization.MalformedRecordException: Successfully parsed a JSON object from input but failed to convert into a Record object with the given schema\n- Caused by: org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Failed Conversion of Field [sharepoint_documents_name__c] from String [10-7-20 2x10GE LAN PHY XXXXXXXXXXXX] to LocalDate\n- Caused by: java.time.format.DateTimeParseException: Text '10-7-20 2x10GE LAN PHY XXXXXXXXXXX' could not be parsed at index 0: Any workaround/fixes available in the newer version. Iam using NIFI 16.1
... View more
Labels:
- Labels:
-
Apache NiFi