Solved

After following this article, I tried to create a flow where email attachments, subject, body, to, from, cc, bcc etc. can be extracted.   However the flow is not proceeding beyond the first processor:ConsumeIMAP is showing 0(0 bytes) in its Out, as shown in screenshot above. 

Hi All, Do we have any tool or utility to mask all the sensitive information (like hostname, ip, etc)  inside the logfiles and Diagnostic bundle before uploading that to Cloudera cases? Also, I can see some of the service logs suffixes with hostname, I need to mask them too. like those below example:- hadoop-cmf-hdfs-NAMENODE-cdp-master.log.out to -> hadoop-cmf-hdfs-NAMENODE-xxx-xxxxx.log.out 

Hi,Recently, in our production environment, there was an abnormal exit of the Hbase Master. In the log, I found that the ZK connection had some problems.According to the sessionid, I found that the sessionid of one of our applications was the same as the sessionid established by the master at the time in question, and then the application side closed the session, and then the master hung. Is this possible? Why does a session with the same ZooKeeper sessionid appear?

Hi, I am very frequently getting the following exception whenever I call the URL seen in the message.  2022-07-06 15:24:56,138 ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.InvokeHTTP InvokeHTTP[id=33351c02-15ee-1ccc-6c00-dda0e16110ca] Routing to Failure due to exception: javax.net.ssl.SSLPeerUnverifiedException: Hostname creatorapp.zoho.com not verified (no certificates): javax.net.ssl.SSLPeerUnverifiedException: Hostname creatorapp.zoho.com not verified (no certificates)javax.net.ssl.SSLPeerUnverifiedException: Hostname creatorapp.zoho.com not verified (no certificates) at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.kt:396) at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.kt:337) at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:209) at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:226) at okhttp3.internal.connection.ExchangeFinder.findHealth

Hi Team ,                 Could you please help me in working with XML tag's in Nifi. I've a scenario where i need to look for the parent xml tag value in entire document, the corresponding child xml tag value should match. if it is not matching we should copy the parent xml tag value to the corresponding specified child tag.here the parent tag is <OriginalQTY name=" 5.000">    and the child tag should match is <OriginalORDEREDPURCHASEQUANTITY>   So every time <OriginalORDEREDPURCHASEQUANTITY> should match <OriginalQTY> value in name. <Document><PURCHPURCHASEORDERHEADERV2ENTITY><VENDORACCOUNT>XXXXX</VENDORACCOUNT><CURRENCYCODE>ZAR</CURRENCYCODE><OriginalQTY name=" 1.000"><PURCHPURCHASEORDERLINEV2ENTITY><PURCHASEORDERNUMBER>PO-XXXXXXX</PURCHASEORDERNUMBER><LINENUMBER>0000000100</LINENUMBER><PURCHASEPRICE&g

Hi Everyone, i am trying to create Hive table with data from mongodb. I have installed mongodb in cloudera quickstart.  loaded json data. sample: { "_id" : ObjectId("59a47286cfa9a3a73e51e749"), "theaterId" : 1034, "location" : { "address" : { "street1" : "3404 W 13th St", "city" : "Grand Island", "state" : "NE", "zipcode" : "68803" }, "geo" : { "type" : "Point", "coordinates" : [ -98.382164, 40.933624 ] } }}  i have create Hive table as below  CREATE EXTERNAL TABLE IF NOT EXISTS THEATER (id STRING,theaterId INT,loc STRUCT<address:STRUCT<street1:STRING,city:STRING,city:STRING,city:STRING,street2:STRING>, geo:STRUCT<type:STRING,coordinates:ARRAY<DOUBLE>> >)STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'WITH SERDEPROPERTIES ('mongo.columns.mapping'='{"id":"_id","theaterId":"theaterId","loc":"location"}')TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/movie_flix.theater'

Hi,how can I alter the table using NIFI Processor where source dbname : org, tableName:status, destination dbName:report and tableName: orgStatus.I am using captureChangeMySql Processor to read the binary-bin log file.Below json i am getting in updateAtribute processor:{"type":"ddl","timestamp":1658078637000,"binlog_filename":"mysql-bin.000030","binlog_position":2727,"database":"nrpsubscriptiondb2306","table_name":null,"table_id":null,"query":"ALTER TABLE `nrpuserorgdb2306`.`status` \nADD COLUMN `is_deleted` TINYINT NULL AFTER `code`"}Please find my processor screenshot for this flow.    

HI,I want NIFI to be run with the minimum resource needed. my concern is whether I can run NIFI with JRE or JDK is also required. will I face any issues in future while scaling up? I am using NIFI 1.15.3.

Hello,I know that there are processors such as CosumIMAP and ConsumPOP3 which can receive email by NiFi. These processors need to provide password.Is there another Processor which utilise Office 365 Oath2 authentification?Considering these two links which show using IMAP will be expired .... https://docs.microsoft.com/en-us/lifecycle/announcements/exchange-online-basic-auth-deprecatedhttps://www.bleepingcomputer.com/news/microsoft/microsoft-warns-exchange-online-basic-auth-will-be-disabled/ Thanks for your supports. 

Hi,I want to manipulate below string in updateAttribute  processor. I tried replace function etc but it is not working for me. Can anyone suggest to modify below string to change DBName and TableName as per expected output.query = ${query:replace('nrpuserorgdb2306.status', 'nrpreportdb.user_org_status')} query : "ALTER TABLE `nrpuserorgdb2306`.`status` \nADD COLUMN `is_deleted` TINYINT NULL AFTER `code`"Expected output:query : ALTER TABLE `nrpreportdb`.`user_org_status` ADD COLUMN `is_test` TINYINT NULL AFTER `code`

I have configured the NiFi flow file where I am reading mysql binary-log file. for me insert, update and delete flow is working fine but when I alter a table then I am getting error.sourceDBName: nrpsubscriptiondb2306 and tableName: product_types destinationDBName: nrpreportdb and tableName: product_types Alter table nifi error:Record had no (or null) value for Field Containing SQL: query, FlowFile StandardFlowFileRecord[uuid=8ca029c9-70aa-4e53-aec3-d37c15c8cd07,claim=StandardContentClaim [resourceClaim=StandardResourceClaim

Hi, I'm trying to make a kafka consumer working, but I am having this issue about SSL Handshake failed. Any ideas ?  2022-07-18 14:00:45,216 INFO [NiFi Web Server-203] o.a.n.c.s.StandardProcessScheduler Starting ConsumeKafkaRecord_2_6[id=f5ee162d-1006-1181-c1d1-1d8a7293ffb7]2022-07-18 14:00:45,217 INFO [NiFi Web Server-203] o.a.n.controller.StandardProcessorNode Starting ConsumeKafkaRecord_2_6[id=f5ee162d-1006-1181-c1d1-1d8a7293ffb7]2022-07-18 14:00:45,217 INFO [Timer-Driven Process Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled ConsumeKafkaRecord_2_6[id=f5ee162d-1006-1181-c1d1-1d8a7293ffb7] to run with 1 threads2022-07-18 14:00:45,219 INFO [Timer-Driven Process Thread-8] o.a.k.clients.consumer.ConsumerConfig ConsumerConfig values:allow.auto.create.topics = trueauto.commit.interval.ms = 5000auto.offset.reset = latestbootstrap.servers = [bootstrap-url:9092]check.crcs = trueclient.dns.lookup = use_all_dns_ipsclient.id = consumer-integration.cubo-transactions-consumer-

Hello everyone you want to install the trial version of CDP private cloud on CentOS 7 machines on an external volume. Can I proceed using this guide?https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/installation/topics/cdpdc-trial-installation.html Or should I follow this?https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/installation/topics/cdpdc-manually-install-cloudera-software-packages.htmlThank you

Hi folks,My Datalake backup of CDP public always fails with this reason: Solr: Could not find any valid local directory for s3ablock-0001-. Anyone knows how to fix?

Hi everyone.I am getting stale alerts every hr, if the last point of contact between Data nodes and Namenode is more than 30 s we get these alerts.I am not able to find the root cause of this slowness, I have 32 cores system, but when this alert is generated in htop hdfs usage is more but not all cores are 100% utilized.DataNode Health SummaryDataNode Health: [Live=5, Stale=1, Dead=0]Please suggest changes required to resolve this.

anyone know why I have thousands and thousands of:2022-07-01 16:28:59,992 INFO org.apache.atlas.AtlasBaseClient: method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=2002022-07-01 16:29:00,017 INFO org.apache.atlas.AtlasBaseClient: method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=2002022-07-01 16:29:00,039 INFO org.apache.atlas.AtlasBaseClient: method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=2002022-07-01 16:29:00,061 INFO org.apache.atlas.AtlasBaseClient: method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=2002022-07-01 16:29:00,085 INFO org.apache.atlas.AtlasBaseClient: method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=2002022-07-01 16:29:00,108 INFO org.ap

I have a flow in which I make a call using InvokeHTTP. The http call returns a JSON response.This JSON response looks like this:{ "token_type": "Bearer", "expires_in": 3599, "ext_expires_in": 3599, "access_token": "eyJ0eXAiOiJKV1QiLCJub25jZSI6InAzVWN1NmRBcTNPRmVGbDM5cTVoZE9EaUpUSFRHT2hVVWdYTUNLMEsyVjQiLCJhbGciOiJSUzI1NiIsIng1dCI6IjJaUXBKM1VwYmpBWVhZR2FYRUpsOGxWMFRPSSIsImtpZCI6IjJaUXBKM1VwYmpBWVhZR2FYRUpsOGxWMFRPSSJ9.eyJhdWQiOiJodHRwczovL2dyYXBoLm1pY3Jvc29mdC5jb20iLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9iZWJhN2VlNi05ZWRhLTQzZTEtOTY5OS0yMDIxYzVmOTI5NDEvIiwiaWF0IjoxNjU3NTUzNzI3LCJuYmYiOjE2NTc1NTM3MjcsImV4cCI6MTY1NzU1NzYyNywiYWlvIjoiRTJaZ1lMalI4cjV6ZWRSMHRxUkErWFdIV2QrZkJnQT0iLCJhcHBfZGlzcGxheW5hbWUiOiJhcHAtZm9yLW5pZmkiLCJhcHBpZCI6ImRlMjg0MjNmLTgxYmMtNDc5Ni1hYTU3LTIyYTM5NmJkNDliYSIsImFwcGlkYWNyIjoiMSIsImlkcCI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0L2JlYmE3ZWU2LTllZGEtNDNlMS05Njk5LTIwMjFjNWY5Mjk0MS8iLCJpZHR5cCI6ImFwcCIsIm9pZCI6IjAwN2EwNjkyLTg5ZjYtNDlmYi04ZDFhLTQwZGE3YjcwZjgwNSIsInJoIjoiMC5BVGdB

I have a huge dataset and I want to split it into partitioned table, but I'm wondering, the second column is date in int and I wonder is it possible to split table based on this, because I don't want to create three more columns with year, month and day. 

Hello all,I do not know how to configure multi users on 3 nodes Nifi Cluster:Currently, i use the following to generate client & server certificates using TLS toolkit:./tls-toolkit.sh standalone -B password -C 'CN=nifiadmin, OU=NIFI' -n 'node1,node2,node3' --nifiDnPrefix 'CN=' --nifiDnSuffix ', OU=NIFI' -o /shared_folder/vm_shared/certs_100722/ -K password -P password -S password I want to configure 3 users for this 3 nodes cluster.Your help would be greatly appreciated.Thank you.

Hi , im unable to connect impala shell ( kerberised cluster ) with below: command  on Edge node impala-shell -i hostname.abc.com -d default -kcase scenario :  we have two Edge node A and B , able to connect on edgenode A  and unable to connect on edgenode  B with same credentials. checked with different user is it also same as working  on Edge node A and edgenode B . observed that both have different python version  A has 2.7.5 as default and B has 3.7.0 is this the cause of connection , if yes how to rectify. ERROR screenshot shared. 

Hello Experts, I am using org.apache.nifi.util.TestRunners in Nifi custom processor unit test cases, and code something like below.. Spoiler (Highlight to read)testRunner = TestRunners.newTestRunner(O9QueueRouting.class);....MockFlowFile ff = this.testRunner.enqueue(new ByteArrayInputStream(ffContents.getBytes(StandardCharsets.UTF_8)));Map<String, String> attrs = new HashMap<String, String>();attrs.put("routingheader", "headervalue");attrs.put("kafka.topic", "topic1");ff.putAttributes(attrs);testRunner.run();testRunner = TestRunners.newTestRunner(O9QueueRouting.class);....MockFlowFile ff = this.testRunner.enqueue(new ByteArrayInputStream(ffContents.getBytes(StandardCharsets.UTF_8)));Map<String, String> attrs = new HashMap<String, String>();attrs.put("routingheader", "headervalue");attrs.put("kafka.topic", "topic1");ff.putAttributes(attrs);testRunner.run();It was working earlier now started giving error like below -[ERROR] O9QueueRoutingTest.test:7

Hello!  Attached below is a flow I have created to perform a simple sorting task with a given CSV file 1. Getfile processor brings the target CSV file into the flow 2. QueryRecord processor executes SQL commands and send the flowfile two ways3. Each flow file is saved into Postgresql database with PutDatabaseRecord processors.  What I am trying to do is. 1. Keep a log of all SQL commands that are executed on Queryrecord processor and keep timestamp (When they were executed) of them 2. Save the log into a Postgresql table named 'validation_log" that has 3 columns. (CSV file name, SQL command, Timestamp) How can I make this work? 

Hi guys,  I need your assistance on querying hbase rest with special characters in the rowkey. Hi, I have the following rowkey and data in Hbase:  ROW COLUMN+CELL \xF2member-00002 column=cf:age, timestamp=2022-07-12T10:16:11.986, value=20    I am querying using the REST interface (version 0.0.3) as:  GET 'http://localhost:8080/member/%F2member-00002'  but always getting Not found.  What should be the correct url encoded rowkey? Thanks!

Change Company Name When I created my registration I chose the wrong option

Hi, I would like to extract a text between two characters for Example @7D1172E5@ using the extract text processor. How do I have to set the setting to get the text between th @ ? Thanks for support