Member since
07-07-2022
42
Posts
2
Kudos Received
8
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4292 | 07-27-2022 12:05 PM | |
1723 | 07-25-2022 05:01 AM | |
926 | 07-21-2022 02:03 AM | |
2221 | 07-21-2022 01:52 AM | |
1367 | 07-19-2022 10:48 AM |
07-22-2022
09:02 AM
Hi , I want to move source DB data to destination DB. Source db table has 'metadat' column, dataType='JSON'. I am using 'captureChangeMySql' processor to ready the change data from mysql binary-log file and put a data to destination db. 'captureChangeMySql' processor to reading the change data from mysql binary-log file but 'metadat' column, dataType='JSON' , value is showing as some encoded format. 1. Can anyone suggest why JSON datatype value is coming encoded like below? 2. Which format is it encoded and how can we get it back as a JSON data to save at destination db. Flowfile output: [ { "id" : 18, "name" : "Transitional Licenses - Provider", "external_id" : "101", "product_tag" : null, "tenant_id" : 1, "metadata" : "\u0000\u0001\u0000I\u0000\u000B\u0000\u000E\u0000\u0002\u0019\u0000mappedProducts\u0004\u00000\u0000\f\u0010\u0000\f\u0018\u0000\f \u0000\f(\u0000\u000720-4040\u000720-4041\u000720-4042\u000720-4043", "duration_in_days" : null, "offline_completion_duration" : null, "product_code" : "cps_provider_7_ed", "external_code" : null, "external_url" : null, "short_description" : "This course is for Providers", "long_description" : "This course is for Providers", "meta_keyword" : null, "meta_description" : null, "meta_title" : null, "status" : "Active", "event_type" : "ESSENTIALS", "event_participation_type" : "REGISTER_ILE", "terms" : null, "product_type_id" : 1, "is_perpetual" : null, "sort_order" : "20", "transitional_sort_order" : null, "completion_cert_type" : null, "ecard_cert_type" : null, "ecard_validity" : null, "ecard_name" : null, "is_deleted" : "{}", "created_at" : "Thu Mar 04 13:15:53 IST 2021", "updated_at" : "Thu Mar 04 13:15:53 IST 2021", "created_by" : null, "updated_by" : null, "is_legacy" : "{0}", "has_ce" : "{}", "region" : "CA", "self_registration_override" : "{}", "allow_assignment" : "{0}", "default_license_type" : null, "dispatch_confirmation" : "{}", "multi_package" : "{}" } ] Original data in db: {"mappedProducts": ["20-4040", "20-4041", "20-4042", "20-4043"]}
... View more
Labels:
- Labels:
-
Apache NiFi
07-20-2022
05:14 AM
@AbhishekSingh I believe you can accomplish this with some very complicated Expression Language chaining. It is possible to have many Expression Languages combined into one expression. Reference: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html I do not have time to prove this out, but in theory, you would combine lesson learned in your other post (expression language to change the database.table with Replace, ifElse and maybe Equals to get your desired outcome.
... View more
07-20-2022
05:02 AM
@AbhishekSingh It looks like the issue with the Replace Expression Language was just the ` around the database and table name. That works for me as follows: ${query:replace('`nrpuserorgdb2306`.`status`', '`nrpreportdb`.`user_org_status`')} Here is my flow to test: Notice the use of a simple flow to test the concept. I also set query in GenerateFlowfile and define a separate query2 in UpdateAttribute. I always work in ways that prove functionality, then when the concept is working, take the lesson learned into real flows. Some additional Screen Shots: Template and Flow Definition File on my GitHub: https://github.com/cldr-steven-matison/NiFi-Templates
... View more
07-19-2022
10:48 AM
Hi , I have modified Sql statement using replace text processor but still getting below error when I alter a table and add new column. PutDatabaseRecord processor input json: { "type" : "ddl", "timestamp" : 1658252209000, "binlog_gtidset" : "e75d07af-eb37-11ec-9d1d-a86daa745b08:1-147", "database" : "nrpsubscriptiondb2306", "table_name" : null, "table_id" : null, "query" : "ALTER TABLE `nrpreportdb`.`user_org_status` ADD COLUMN `is_test` VARCHAR(255) NULL AFTER `code`" } Error log: Record had no (or null) value for Field Containing SQL: query, FlowFile StandardFlowFileRecord[uuid=bc7d51b3-e71f-4eb2-834b-fecbcff572be,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1658226664320-3, container=default, section=3], offset=37727, length=277],offset=0,name=bc7d51b3-e71f-4eb2-834b-fecbcff572be,size=277]
... View more
07-17-2022
11:41 PM
@AbhishekSingh, as this is an older article, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this article as a reference in your new post.
... View more
07-17-2022
11:15 AM
Hi, I want to create and validate a schema registry dynamically. Can anyone suggest a right approach to do it. Attaching my Nifi screenshot for reference .
... View more
Labels:
- Labels:
-
Apache NiFi
07-14-2022
12:44 AM
Hi, How can i define the Initial Binlog Position in captureChangeMySql processor configuration in expression language to read the binlog greater than or equal given number.
... View more
Labels:
- Labels:
-
Apache NiFi
07-13-2022
09:41 AM
@araujo Can you please suggest how I can fix above issue. CaptureChangeMySql processor is giving value "{}" for BIT datatype. Processor out payload: { "type": "update", "timestamp": 1657194900000, "binlog_filename": "mysql-bin.000020", "binlog_position": 37029, "database": "nrpsubscriptiondb2306", "table_name": "user_subscription", "table_id": 137, "columns": [ { "id": 29, "name": "is_deleted", "column_type": -7, "last_value": "{}", "value": "{}" }, ] }
... View more
07-13-2022
06:42 AM
@AbhishekSingh , You can use the following expression instead to format the date as you want in the UpdateRecord processor: /subscription_start_at_timestamp -> format(toDate(/subscription_start_at, "EEE MMM d HH:mm:ss z yyyy"), "yyyy-MM-dd HH:mm:ss") Cheers, André
... View more
- « Previous
-
- 1
- 2
- Next »