Hi,   I want to connect  connect two different MySQL database services(mysql 5.7 and Mysql 8.0 ). I am using two  two CaptureChangeMySQL Processors for this.  Mysql 5.7 is install in my local machine and CaptureChangeMySQL Processors connecting successfully.   Mysql 8.0 is install on docker container and on host machine port 3310 like (sudo docker run --name mysql-8.0-container -p 127.0.0.1:3310:3306 -e MYSQL_ROOT_PASSWORD=password -d mysql:8.0).  CaptureChangeMySQL connecting with mysql 8.0 is giving  error.   Can any suggest why i am getting this error and how to resolve it. Note: I am able to connect mysql 8.0 using mysql workbench.     Error: 2022-08-03 15:59:58,863 ERROR [Timer-Driven Process Thread-4] o.a.n.c.m.processors.CaptureChangeMySQL CaptureChangeMySQL[id=352daba9-0182-1000-a02a-faf160ab1329] Processing failed org.apache.nifi.processor.exception.ProcessException: Could not connect binlog client to any of the specified hosts due to: Client does not support authentication protocol requested by server; consider upgrading MySQL client at org.apache.nifi.cdc.mysql.processors.CaptureChangeMySQL.setup(CaptureChangeMySQL.java:664) at org.apache.nifi.cdc.mysql.processors.CaptureChangeMySQL.onTrigger(CaptureChangeMySQL.java:678) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1283) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)      Thanks,     ... View more

Hi, I am getting error "Record does not have a value for the PrimaryKey column 'HOST' " when I update a data in source db table. This issue is happening when destination DB table's name: 'user' or 'users'. It is working fine when destination table's name is change to 'report_user' (I mean when we change the table name 'user'or 'users' to something else which is not used as mysql internal  table)  I am not sure why it is trying to insert or update on internal mysql table as I am giving the destination DBName and tableName. Can anyone suggest how can we fix this issue without changing the destination table name?     Error Log: PutDatabaseRecord[id=ff8ca68c-8252-3652-8f55-1044ad3f1bab] Failed to put Records to database for StandardFlowFileRecord[uuid=43e64992-a3d0-4a75-bb85-8d70c1f556d7,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1658747398462-4, container=default, section=4], offset=20549, length=1214],offset=0,name=43e64992-a3d0-4a75-bb85-8d70c1f556d7,size=1214]. Routing to failure. org.apache.nifi.serialization.MalformedRecordException: Record does not have a value for the PrimaryKey column 'HOST' at org.apache.nifi.processors.standard.PutDatabaseRecord.normalizeKeyColumnNamesAndCheckForValues(PutDatabaseRecord.java:1367)   PutDatabaseRecord processor flow file output: [ { "id" : 1, "first_name" : "john", "middle_name" : "s_test", "last_name" : "asjndjnas", "user_name" : "deleted_8AB-0034", "is_terms_accepted" : "true", "is_retail" : "true", "tenant_id" : 1, "email" : "sreejith.karonnan2@mailinator.com", "alternate_email" : null, "gender" : "M", "dob" : "2008-01-01", "sso_id" : null, "phone_number" : null, "status_id" : 2, "time_zone_id" : 1, "is_activated" : "true", "activated_at" : "2021-01-07 16:29:25", "is_deleted" : "false", "import_id" : null, "created_at" : "2021-01-07 16:29:25", "updated_at" : "2021-11-16 18:38:42", "created_by" : 123, "updated_by" : 3, "last_login_date" : null, "agreement_acceptance_date" : "2021-03-22 20:44:23", "user_ref_id" : "589RngS1", "is_admin" : "false", "is_distributor" : "true", "is_self_register" : "false", "ce_job_id" : null, "region" : "US", "recovery_status" : null, "hlc_user_name" : null, "allow_assignment" : false, "has_ce" : false, "product_launched_at" : null, "is_perpetual" : false, "learning_completed_at" : null, "enrolled_at" : null, "dispatch_confirmation" : false, "subscription_end_at" : null, "ecard_date" : null, "offline_completion_date" : null, "subscription_start_at" : null, "product_completed_at" : null, "online_completion_date" : null, "is_legacy" : false, "last_activity_at" : null, "multi_package" : false, "metadata" : null, "self_registration_override" : false } ] Thanks, ... View more

Hi, I have install a NiFi setup and deign the flow file to move data from source mysql db to destination mysql db. Now I want to know how can I setup it in dev or QA environments. I was going to NiFi admin guide and came to know about NiFi Registry and NiFi toolkit feature. I have below doubts: 1. Is it mandatory  to install NiFi toolkit along with NiFi to configure clustering and managing the cluster node? or without installing a NiFi toolkit, we can configure cluster and manage it?  2.  Is it mandatory  to install NiFi Registry along with NiFi to manage the flowfile change restriction?   ... View more

Hi, Query 1: How to connect to two different source DB instants using captureChangeMySql processor?. Query 2: is it possible to configure two different source DB instants using captureChangeMySql processor to get data from both db instance and put it in destination DB?, If yes, Please suggest the configuration approach. Below is my current configuration screenshot to read the data from single instant connecting two source DB.   ... View more

Hi, I want to manipulate below string in updateAttribute  processor. I tried replace function etc but it is not working for me. Can anyone suggest to modify below string to change DBName and TableName as per expected output. query = ${query:replace('nrpuserorgdb2306.status', 'nrpreportdb.user_org_status')}   query : "ALTER TABLE `nrpuserorgdb2306`.`status` \nADD COLUMN `is_deleted` TINYINT NULL AFTER `code`" Expected output: query : ALTER TABLE `nrpreportdb`.`user_org_status` ADD COLUMN `is_test` TINYINT NULL AFTER `code` ... View more

I have configured the NiFi flow file where I am reading mysql binary-log file. for me insert, update and delete flow is working fine but when I alter a table then I am getting error. sourceDBName: nrpsubscriptiondb2306 and tableName: product_types   destinationDBName: nrpreportdb and tableName: product_types   Alter table nifi error: Record had no (or null) value for Field Containing SQL: query, FlowFile StandardFlowFileRecord[uuid=8ca029c9-70aa-4e53-aec3-d37c15c8cd07,claim=StandardContentClaim [resourceClaim=StandardResourceClaim ... View more

Hi, how can I alter the table using NIFI Processor where source dbname : org, tableName:status, destination dbName:report and tableName: orgStatus. I am using captureChangeMySql Processor to read the binary-bin log file. Below json i am getting in updateAtribute processor: {"type":"ddl","timestamp":1658078637000,"binlog_filename":"mysql-bin.000030","binlog_position":2727,"database":"nrpsubscriptiondb2306","table_name":null,"table_id":null,"query":"ALTER TABLE `nrpuserorgdb2306`.`status` \nADD COLUMN `is_deleted` TINYINT NULL AFTER `code`"} Please find my processor screenshot for this flow.        ... View more

Hi,  I want to convert some of the field value like  "enrolled_at":"Sun Oct 17 15:58:58 IST 2021" -> DateTime(2021-10-17 15:58:58) and "is_deleted":"{0}" -> boolean(true or false) from below Json array to save converted data in db. How i can achieve it in NiFi.   [{"id":234521,"tenant_id":1,"product_id":1,"version":"3","user_id":197172,"user_ref_id":"8TS-0516","org_id":910,"org_subscription_id":7294,"access_code":null,"order_ref_id":null,"access_code_id":null,"enrol_type":"Admin4","enrolled_at":"Sun Oct 17 15:58:58 IST 2021","subscription_start_at":"Wed Oct 13 15:58:58 IST 2021","subscription_end_at":"Sat Oct 23 15:58:58 IST 2021","product_launched_at":"Wed Oct 13 16:05:49 IST 2021","product_completed_at":"Wed Oct 13 16:15:44 IST 2021","learning_completed_at":"Wed Oct 13 16:15:44 IST 2021","subscription_status":"ACTIVE","learning_status":"COMPLETED","total_credits":null,"completion_certificate_id":3585,"ecard_id":35867,"certificate_number":"tocubg3sfbkzfgfqslnisnci","ecard_number":"1hr5w9jkpikonazrfhqquauh","online_completion_date":"Wed Oct 13 16:08:07 IST 2021","offline_completion_date":"Wed Oct 13 16:15:44 IST 2021","ecard_date":"Tue Oct 31 16:15:44 IST 2023","is_deleted":"{0}","created_by":1,"updated_by":14302,"org_Name":"International Society","assignment_id":961,"last_activity_at":"Wed Oct 13 16:15:44 IST 2021","updated_at":"Wed Oct 13 16:15:44 IST 2021","source_lms":null,"course_instance_id":"20-3583"}] ... View more