Member since
01-26-2022
82
Posts
2
Kudos Received
5
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1213 | 09-08-2022 04:02 AM | |
| 2427 | 07-27-2022 03:29 AM | |
| 1863 | 06-28-2022 09:43 PM | |
| 1886 | 06-27-2022 08:42 AM | |
| 18629 | 06-16-2022 10:29 PM |
04-16-2023
01:03 AM
Hello, I am using jolt to change the structure of a payload to another form, and it must be dynamic, especially the top level elements "test" which can be multiple. I have tried but not getting it. I would appreciate your kind help. Here is my input json: {
"test1": "test output",
"test2": "test output",
"Details": {
"0": {
"id": "first",
"name": "the first one"
},
"1": {
"id": "second",
"name": "the second one"
}
}
} and my output json is : {
"Details" : [ {
"test1" : "test output",
"test2" : "test output",
"id" : "first",
"name" : "the first one"
}, {
"test1" : "test output",
"test2" : "test output",
"id" : "second",
"name" : "the second one"
} ]
} my current jolt spec is: [
{
"operation": "shift",
"spec": {
"Details": "Details",
"*": "outer.&"
}
},
{
"operation": "shift",
"spec": {
"Details": {
"*": {
"*": "Details[&1].&",
"@(2,outer)": "Details[&1].outer"
}
}
}
},
{
"operation": "shift",
"spec": {
"Details": {
"*": {
"outer": {
"*": "Details[&2].&"
},
"*": "Details[&1].&"
}
}
}
}
] But I am not getting the desired output json as above.
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
03-09-2023
08:49 AM
As @SAMSAL pointed out, queryrecord is the best and easiest method to use. Here, i have created a sample flow for your use as pix attached. I hope it helps.
... View more
09-28-2022
04:47 AM
Thank you all. I eventually evaluated the json path to extract the url. My mind was astray as i was using complex solution to a simple problem.
... View more
09-25-2022
10:45 PM
Hello, I am trying to extract value from a key/value pair json record. I am trying to extract the value of "image" key. I used an ExtractText processor but there was no match. Although, there was a match when i used an online regex extractor(https://onlinetexttools.com/extract-regex-matches-from-text). Regex i used in the extracttext processor: (?<=\"image\"\s:\s")[A-Z-a-z-0-9\-\:\/\.\_]+ My json record: {
"id" : "03ee73b8-a553-4575-8dfa-d0da4e7939e9",
"url" : "https://ll.thespacedevs.com/2.0.0/launch/03ee73b8-a553-4575-8dfa-d0da4e7939e9/",
"launch_library_id" : null,
"slug" : "falcon-9-block-5-galaxy-33-34",
"name" : "Falcon 9 Block 5 | Galaxy 33 & 34",
"status" : {
"id" : 2,
"name" : "TBD"
},
"net" : "2022-10-05T23:07:00Z",
"window_end" : "2022-10-06T00:14:00Z",
"window_start" : "2022-10-05T23:07:00Z",
"inhold" : false,
"tbdtime" : false,
"tbddate" : false,
"probability" : null,
"holdreason" : "",
"failreason" : "",
"hashtag" : null,
"launch_service_provider" : {
"id" : 121,
"url" : "https://ll.thespacedevs.com/2.0.0/agencies/121/",
"name" : "SpaceX",
"type" : "Commercial"
},
"rocket" : {
"id" : 7549,
"configuration" : {
"id" : 164,
"launch_library_id" : 188,
"url" : "https://ll.thespacedevs.com/2.0.0/config/launcher/164/",
"name" : "Falcon 9",
"family" : "Falcon",
"full_name" : "Falcon 9 Block 5",
"variant" : "Block 5"
}
},
"mission" : {
"id" : 5976,
"launch_library_id" : null,
"name" : "Galaxy 33 & 34",
"description" : "Galaxy 33, 34 are two geostationary communications satellites manufactured by Northrop Grumman and operated by Intelsat.",
"launch_designator" : null,
"type" : "Communications",
"orbit" : {
"id" : 2,
"name" : "Geostationary Transfer Orbit",
"abbrev" : "GTO"
}
},
"pad" : {
"id" : 80,
"url" : "https://ll.thespacedevs.com/2.0.0/pad/80/",
"agency_id" : 121,
"name" : "Space Launch Complex 40",
"info_url" : null,
"wiki_url" : "https://en.wikipedia.org/wiki/Cape_Canaveral_Air_Force_Station_Space_Launch_Complex_40",
"map_url" : "http://maps.google.com/maps?q=28.56194122,-80.57735736",
"latitude" : "28.56194122",
"longitude" : "-80.57735736",
"location" : {
"id" : 12,
"url" : "https://ll.thespacedevs.com/2.0.0/location/12/",
"name" : "Cape Canaveral, FL, USA",
"country_code" : "USA",
"map_image" : "https://spacelaunchnow-prod-east.nyc3.digitaloceanspaces.com/media/launch_images/location_12_20200803142519.jpg",
"total_launch_count" : 858,
"total_landing_count" : 24
},
"map_image" : "https://spacelaunchnow-prod-east.nyc3.digitaloceanspaces.com/media/launch_images/pad_80_20200803143323.jpg",
"total_launch_count" : 154
},
"webcast_live" : false,
"image" : "https://spacelaunchnow-prod-east.nyc3.digitaloceanspaces.com/media/launcher_images/falcon_9_block__image_20210506060831.jpg",
"infographic" : null,
"program" : [ ]
} Expected output: https://spacelaunchnow-prod-east.nyc3.digitaloceanspaces.com/media/launcher_images/falcon_9_block__image_20210506060831.jpg Thanks for your help.
... View more
Labels:
09-15-2022
03:34 AM
Hello, Please we have been having issues recently where we discover nifi is not clearing queue. The que is not going up or going down. And the ERROR IS: "unable to save flow configuration" The java version is 11.0.4 Platform: RHEL 8 Nifi: 1.15.3 Please can you kindly look into it. Thanks.
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
09-08-2022
04:02 AM
After fetching from the first DB, you can convert each record to json, the use evaluatejsonpath to extract the timestamp as attribute. then use updateAttribute processor to update timestamp attribute you extracted earlier: Let's say on evaluatejsonpath, the extracted timestamp attribute is "dTime". Then , on updateAttribute, you create same atttribute and set dTime value to ${dTime:toDate("yyyy-MM-dd hh:mm:ss"):format("yyyy-MM-dd hh:mm:ss.SSSSSS")}
... View more
09-06-2022
02:44 PM
Thanks for the insight Mr @araujo . I would like to tell you i have thousands of that XML files. Please is there any better way of doing it. Thank you.
... View more
09-06-2022
01:20 PM
1 Kudo
If you have access to the Oracle Db, create a View for your query on that DB. Then use the View name as Db name inside QueryDatabaseTable. NOTE: If you have complex query, do use View instead. I hope this helps.
... View more
09-06-2022
01:13 PM
Hi,
Please i need your kind help.
SCENERIO: INSERT CONTENT OF AN XML FILE AS A FIELD INTO A DATABASE
- I am trying to read the content of an xml file into a flowfile attribute
- I want to use the attribute as variable in sql query statement to insert the content into a DB
ISSUES:
- I tried to use both Xpath & Xquery but no success.
DATA: The data i am trying to convert is as below
<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE database>
<database xmlns='http://www.lotus.com/dxl' version='9.0' maintenanceversion='1.0'
replicaid='8525886700702AFD' path='BStravel2.nsf' title='BSTravel_Expenses_and_Vacations'
maintainunread='false' optimizetablebitmaps='true' increasemaxfields='true'
compressdesign='true' compressdata='true' markmodifiedunread='false' logtransactions='false'
uselz1='true'>
<databaseinfo dbid='85256700702AFD' odsversion='43' diskspace='261440'
percentused='96.2421875' numberofdocuments='13'><datamodified><datetime
dst='true'>20220813253,61-04</datetime></datamodified><designmodified
><datetime dst='true'>2022062936,80-04</datetime></designmodified></databaseinfo>
<launchsettings><noteslaunch whenopened='openframeset' frameset='Launch' showaboutdocument='never'/></launchsettings></database>
Thank you so much for all your help .
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
09-04-2022
01:08 PM
Thanks Mr @SAMSAL for the insight. I learned from it because i was even thinking of telling him to use EvaluateJsonPath & ReplaceText processor.
... View more