Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Getting Duplicate records in MySQL database

avatar
Contributor

64527-screen-shot-2018-03-07-at-50438-pm.png

I'm trying to copy the data from one database(MySQL) to another database(MySQL). Below is the flow.

GenerateTableFetch Partition Size : 1500 records. Total 3200 records so it's creating 3 files as shown in the picture.

64532-main.png

I'm able to read the data from the source database(avro format) and convert the data to json(using AvrotoJSON processor). Now I'm trying to load the JSON data into MySQL table using ConvertJSONTOSQL Pricessor. In the source table I have 3200

records, however in my destination table I'm getting 10000+ records. I didn't get where I did mis configuration so that it's inserting duplicate records into destination table. Below are screenshots of the individual processors.

Any suggestions please.


putsql.pngexecute-sql.pngconvertavrotojson.pngconvertjsontosql.png
1 ACCEPTED SOLUTION

avatar
Master Guru
@User 805

You need to autoterminate original relationship in ConvertJSONtoSQL processor, now original relationship is loop back into same processor again which is causing all the duplicates in mysql database.

62696-autoterminate.png

How to Autoterminate original relationship:-

  1. clear the queue first.
  2. delete the original relationship
  3. right click on processor
  4. goto setting tab
  5. click on checkbox at original relationship as shown in the below screenshot

62697-autoterminate-steps.png

.

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.

View solution in original post

2 REPLIES 2

avatar
Master Guru
@User 805

You need to autoterminate original relationship in ConvertJSONtoSQL processor, now original relationship is loop back into same processor again which is causing all the duplicates in mysql database.

62696-autoterminate.png

How to Autoterminate original relationship:-

  1. clear the queue first.
  2. delete the original relationship
  3. right click on processor
  4. goto setting tab
  5. click on checkbox at original relationship as shown in the below screenshot

62697-autoterminate-steps.png

.

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.

avatar
Contributor

Hi @Shu,

It worked like charm. Thanks a lot for the quick response.