Member since
01-26-2022
70
Posts
2
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
133 | 09-08-2022 04:02 AM | |
207 | 07-27-2022 03:29 AM | |
312 | 06-28-2022 09:43 PM | |
251 | 06-27-2022 08:42 AM | |
2059 | 06-16-2022 10:29 PM |
10-14-2022
12:35 AM
Hello All, I would like to know if there is way i can send notification from Nifi to WhatsApp platform. Thank you.
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
09-28-2022
04:47 AM
Thank you all. I eventually evaluated the json path to extract the url. My mind was astray as i was using complex solution to a simple problem.
... View more
09-25-2022
10:45 PM
Hello, I am trying to extract value from a key/value pair json record. I am trying to extract the value of "image" key. I used an ExtractText processor but there was no match. Although, there was a match when i used an online regex extractor(https://onlinetexttools.com/extract-regex-matches-from-text). Regex i used in the extracttext processor: (?<=\"image\"\s:\s")[A-Z-a-z-0-9\-\:\/\.\_]+ My json record: {
"id" : "03ee73b8-a553-4575-8dfa-d0da4e7939e9",
"url" : "https://ll.thespacedevs.com/2.0.0/launch/03ee73b8-a553-4575-8dfa-d0da4e7939e9/",
"launch_library_id" : null,
"slug" : "falcon-9-block-5-galaxy-33-34",
"name" : "Falcon 9 Block 5 | Galaxy 33 & 34",
"status" : {
"id" : 2,
"name" : "TBD"
},
"net" : "2022-10-05T23:07:00Z",
"window_end" : "2022-10-06T00:14:00Z",
"window_start" : "2022-10-05T23:07:00Z",
"inhold" : false,
"tbdtime" : false,
"tbddate" : false,
"probability" : null,
"holdreason" : "",
"failreason" : "",
"hashtag" : null,
"launch_service_provider" : {
"id" : 121,
"url" : "https://ll.thespacedevs.com/2.0.0/agencies/121/",
"name" : "SpaceX",
"type" : "Commercial"
},
"rocket" : {
"id" : 7549,
"configuration" : {
"id" : 164,
"launch_library_id" : 188,
"url" : "https://ll.thespacedevs.com/2.0.0/config/launcher/164/",
"name" : "Falcon 9",
"family" : "Falcon",
"full_name" : "Falcon 9 Block 5",
"variant" : "Block 5"
}
},
"mission" : {
"id" : 5976,
"launch_library_id" : null,
"name" : "Galaxy 33 & 34",
"description" : "Galaxy 33, 34 are two geostationary communications satellites manufactured by Northrop Grumman and operated by Intelsat.",
"launch_designator" : null,
"type" : "Communications",
"orbit" : {
"id" : 2,
"name" : "Geostationary Transfer Orbit",
"abbrev" : "GTO"
}
},
"pad" : {
"id" : 80,
"url" : "https://ll.thespacedevs.com/2.0.0/pad/80/",
"agency_id" : 121,
"name" : "Space Launch Complex 40",
"info_url" : null,
"wiki_url" : "https://en.wikipedia.org/wiki/Cape_Canaveral_Air_Force_Station_Space_Launch_Complex_40",
"map_url" : "http://maps.google.com/maps?q=28.56194122,-80.57735736",
"latitude" : "28.56194122",
"longitude" : "-80.57735736",
"location" : {
"id" : 12,
"url" : "https://ll.thespacedevs.com/2.0.0/location/12/",
"name" : "Cape Canaveral, FL, USA",
"country_code" : "USA",
"map_image" : "https://spacelaunchnow-prod-east.nyc3.digitaloceanspaces.com/media/launch_images/location_12_20200803142519.jpg",
"total_launch_count" : 858,
"total_landing_count" : 24
},
"map_image" : "https://spacelaunchnow-prod-east.nyc3.digitaloceanspaces.com/media/launch_images/pad_80_20200803143323.jpg",
"total_launch_count" : 154
},
"webcast_live" : false,
"image" : "https://spacelaunchnow-prod-east.nyc3.digitaloceanspaces.com/media/launcher_images/falcon_9_block__image_20210506060831.jpg",
"infographic" : null,
"program" : [ ]
} Expected output: https://spacelaunchnow-prod-east.nyc3.digitaloceanspaces.com/media/launcher_images/falcon_9_block__image_20210506060831.jpg Thanks for your help.
... View more
Labels:
09-15-2022
03:34 AM
Hello, Please we have been having issues recently where we discover nifi is not clearing queue. The que is not going up or going down. And the ERROR IS: "unable to save flow configuration" The java version is 11.0.4 Platform: RHEL 8 Nifi: 1.15.3 Please can you kindly look into it. Thanks.
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
09-08-2022
12:41 PM
Hi All, Please can you kindly look into this issue i have with QueryNiFiReportingTask processor. PLATFORM: - WSL2 with Ubuntu 20.04 as OS - jdk-11.0.15 - NIFI 1.17.0 SCENERIO: I am trying to monitor all processors so as to know which one has stopped & sending notification through email sink. I configured gmail email sender appropriately. ISSUES: No email received & there was no error on the bulleting indicating failure. NOTE: I changed smtp to different ones but got errors. Also enabling ssl gives me error. Kindly find the pix below for my configurartions.
... View more
Labels:
09-08-2022
04:02 AM
After fetching from the first DB, you can convert each record to json, the use evaluatejsonpath to extract the timestamp as attribute. then use updateAttribute processor to update timestamp attribute you extracted earlier: Let's say on evaluatejsonpath, the extracted timestamp attribute is "dTime". Then , on updateAttribute, you create same atttribute and set dTime value to ${dTime:toDate("yyyy-MM-dd hh:mm:ss"):format("yyyy-MM-dd hh:mm:ss.SSSSSS")}
... View more
09-06-2022
02:44 PM
Thanks for the insight Mr @araujo . I would like to tell you i have thousands of that XML files. Please is there any better way of doing it. Thank you.
... View more
09-06-2022
01:20 PM
1 Kudo
If you have access to the Oracle Db, create a View for your query on that DB. Then use the View name as Db name inside QueryDatabaseTable. NOTE: If you have complex query, do use View instead. I hope this helps.
... View more
09-06-2022
01:13 PM
Hi,
Please i need your kind help.
SCENERIO: INSERT CONTENT OF AN XML FILE AS A FIELD INTO A DATABASE
- I am trying to read the content of an xml file into a flowfile attribute
- I want to use the attribute as variable in sql query statement to insert the content into a DB
ISSUES:
- I tried to use both Xpath & Xquery but no success.
DATA: The data i am trying to convert is as below
<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE database>
<database xmlns='http://www.lotus.com/dxl' version='9.0' maintenanceversion='1.0'
replicaid='8525886700702AFD' path='BStravel2.nsf' title='BSTravel_Expenses_and_Vacations'
maintainunread='false' optimizetablebitmaps='true' increasemaxfields='true'
compressdesign='true' compressdata='true' markmodifiedunread='false' logtransactions='false'
uselz1='true'>
<databaseinfo dbid='85256700702AFD' odsversion='43' diskspace='261440'
percentused='96.2421875' numberofdocuments='13'><datamodified><datetime
dst='true'>20220813253,61-04</datetime></datamodified><designmodified
><datetime dst='true'>2022062936,80-04</datetime></designmodified></databaseinfo>
<launchsettings><noteslaunch whenopened='openframeset' frameset='Launch' showaboutdocument='never'/></launchsettings></database>
Thank you so much for all your help .
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
09-04-2022
01:08 PM
Thanks Mr @SAMSAL for the insight. I learned from it because i was even thinking of telling him to use EvaluateJsonPath & ReplaceText processor.
... View more
08-15-2022
12:24 PM
Hi all. I am using PutEmail processor to send email when one of my core processors failed to process. It has been working before but it has stopped working due to this Google Security update Any help with this? I am using gmail on NIFI 1.15.3 NOTE: All my credentials are correct Thank you all. ERROR: PutEmail[id=d90f3185-017e-1000-33a0-5dfbaf747749] Failed to send email for StandardFlowFileRecord[uuid=0a6df4d8-7cfc-4d05-9261-304d178d0834,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1660589165172-1, container=default, section=1], offset=1877, length=87],offset=0,name=0a6df4d8-7cfc-4d05-9261-304d178d0834,size=87]: 535-5.7.8 Username and Password not accepted. Learn more at
535 5.7.8 https://support.google.com/mail/?p=BadCredentials co26-20020a0560000a1a00b0021e2fccea97sm8120496wrb.64 - gsmtp
; routing to failure: 535-5.7.8 Username and Password not accepted. Learn more at
535 5.7.8 https://support.google.com/mail/?p=BadCredentials co26-20020a0560000a1a00b0021e2fccea97sm8120496wrb.64 - gsmtp Google Security Update: Thanks to you all.
... View more
Labels:
08-10-2022
04:11 PM
Hello, please kindly help. i am trying to convert a datetime(attribute) from one format to the other but it keeps giving me empty value on the attribute: SCENERIO: Flowfile is json----->extract date from Json flowfile using Evaluatejsonpath---->format date using Updateattribute Actual Date: "2022-08-01 00:00:00.0" Desired date: "01/08/2022 00:00:00.000000" I have tried to use this on the updateattribute, but it is not working: ${DATE_TIME:toNumber():toDate('yyyy-MM-dd hh:mm:ss.S'):format('dd/MM/yyyy HH:mm:ss.SSSSSS')} Thank you.
... View more
Labels:
07-27-2022
03:29 AM
Thanks everyone. I have solved the problem by including dependency on the pom.xml files of nifi-custom-processors & nifi-custom-nar. The dependency is like this. < dependency > < groupId > commons-io </ groupId > < artifactId > commons-io </ artifactId > < version > 2.11.0 </ version > </ dependency >
... View more
07-26-2022
01:53 PM
Hello, Please kindly help to resolve this error. I am trying to learn how to create a Nifi custom processor following this tutorial: https://help.syncfusion.com/data-integration/how-to/create-a-custom-processor#_Project_Creation%22%22 But i don't know the class to import to solve the below error: Below are all the imports i have on the file import org.apache.nifi.components.PropertyDescriptor; import org.apache.nifi.flowfile.FlowFile; import org.apache.nifi.annotation.behavior.ReadsAttribute; import org.apache.nifi.annotation.behavior.ReadsAttributes; import org.apache.nifi.annotation.behavior.WritesAttribute; import org.apache.nifi.annotation.behavior.WritesAttributes; import org.apache.nifi.annotation.lifecycle.OnScheduled; import org.apache.nifi.annotation.documentation.CapabilityDescription; import org.apache.nifi.annotation.documentation.SeeAlso; import org.apache.nifi.annotation.documentation.Tags; import org.apache.nifi.processor.AbstractProcessor; import org.apache.nifi.processor.ProcessContext; import org.apache.nifi.processor.ProcessSession; import org.apache.nifi.processor.ProcessorInitializationContext; import org.apache.nifi.processor.Relationship; import org.apache.nifi.processor.util.StandardValidators; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import org.apache.nifi.processor.io.StreamCallback; import org.apache.nifi.processor.exception.ProcessException; import java.util.ArrayList; import java.util.Collections; import java.util.HashSet; import java.util.List; import java.util.Set; Thanks for your kind help
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
07-18-2022
01:01 PM
Thanks for your help. I also chatted with one of Streamnative dev advocates & he directed me to tons of info. Unfortunately, the nars are & didn't work with on prem Nifi 1.16.3. Also, i tried to build from source files but encountered lot of errors.
... View more
07-18-2022
10:26 AM
Hi, I am trying to integrate Nifi with Pulsar. But unfortunately i cannot find any of Pulsar nar Processors around. I am running Nifi 1.16.3 Thanks for your help.
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
07-18-2022
10:21 AM
Have you checked this link: https://community.cloudera.com/t5/Support-Questions/Adding-columns-to-sql-in-nifi/m-p/224598
... View more
07-14-2022
09:33 AM
Thank you sir. I was able to generate more users/client certificates by specifying multiple "-C CN=user1, OU=nifi" in addition to the initial command i have.
... View more
07-12-2022
01:01 PM
As @SAMSAL pointed out kindly checked the attached xml: 1) Get the csv file 2) query the csv file for null & not null using query record. The queries are represented by variables. 3) the null & not null variables are used for routing. I hope it helps. get xml from here: https://drive.google.com/file/d/1DjrW46GCRlVUTVGRP1HXRVKOREWCKUWe/view?usp=sharing
... View more
07-12-2022
10:03 AM
Thanks Mr @MattWho . In addition to your solution, i looked into Network Attached Storage & NFS. So i configured nfs server on the source, then configured nfs client on the destination. And lastly mount the source folder to the destination folder on restart. Thank you all.
... View more
07-12-2022
09:36 AM
Hello all, I do not know how to configure multi users on 3 nodes Nifi Cluster: Currently, i use the following to generate client & server certificates using TLS toolkit: ./tls-toolkit.sh standalone -B password -C 'CN=nifiadmin, OU=NIFI' -n 'node1,node2,node3' --nifiDnPrefix 'CN=' --nifiDnSuffix ', OU=NIFI' -o /shared_folder/vm_shared/certs_100722/ -K password -P password -S password I want to configure 3 users for this 3 nodes cluster. Your help would be greatly appreciated. Thank you.
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
07-08-2022
06:36 AM
Please, i have files i want to move from a Nifi to a folder in another machine. The ip address of Nifi & the destination are 173.24.206.40 ,173.24.206.44 respectively, meaning they are on the same network. I specified 173.24.206.44/home/username/destination_folder as distination directory in PUTFile processor BUT the destination_folder is empty & there was no error. Thanks.
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
06-29-2022
01:22 PM
Thank you Sir for the reply. Yes i have looked into them. The issue i have is that my query statement has multiple join conditions. And also the table is incremental. So, someone proposed i should convert the query to a database table view, then i can now supply the view name to any of the processors you mentioned above. Please, do you have a better approach. Thank you sir.
... View more
06-29-2022
11:28 AM
Thank you so much Mr @MattWho for comprehensive explanation on what Mr @SAMSAL earlier proposed. I really learn alot & appreciate.
... View more
06-28-2022
09:43 PM
1 Kudo
I finally got it. I copied one of the standalone truststore.jks & keystore.jks i generated for the server nodes. Then pasted into the conf folder of the Minifi. Also, i updated the yml file with the password( the one i used when generating the certificate) & the path to the truststore.jks & keystore.jks. So, everything works perfectly. Thank you.
... View more
06-28-2022
09:34 PM
Thank you sir. I thought same way as much. I was just thinking may be there is a better way to go about it. Please, do you have any other method?
... View more
06-28-2022
10:09 AM
Hi, I have 3-node Nifi cluster. Please, i would like to ask if each node of the cluster can output different results (i.e no duplicates) from one another. SCENARIO: Let say i have 10 records on a DB. If node1 output the first record, i want node2 or node3 not to output first record again. Thank you.
... View more
Labels:
06-28-2022
06:47 AM
Thanks. I think your approach is better to push all overheads to the Oracle Db rather than NIFI. 👍
... View more