Support Questions

Find answers, ask questions, and share your expertise

PutHiveQL executed the same query twice!!!

avatar
New Contributor

Hello there.. 
I am using replacetext processor where i am using the below query as my replacement value.

INSERT into log_hive.log_ozone_retention_jobs
(Table_Name,Process_Date, Process ,Status ,Source_Count ,Target_Count ,Start_Time ,End_Time ,Notes)
select 'Customer_Voip',${date_list}, 'Insert_Into_Ozone','Insert Successful',${source_count},null,'${start_time}',current_timestamp(),null;

date_list is an attribute value coming along with the incoming flowfile.. 

My downstream processor is PutHiveQL where it executes the insert query provided above..  I was successfully able to insert a single record for that day using the  PutHiveQL processor.. However, today i found that PutHiveQL actually inserted 2 records for the same day as below... the incoming flowfile had date_list value as 20250401 for example and is expected to insert only 1 record but inserted 2 records with 2 different end time... this happened with all the dates that were meant to be loaded...
Rashad_K_1-1759845915895.png

PutHiveQL processor configuration..

Rashad_K_2-1759846022776.pngRashad_K_3-1759846058034.png

Interestingly, I have used the same processor in the next process group and doing the same insert but for other purpose and as per expectations it has inserted only 1 record per date_list/process_date.. and just to inform that i am using the similar logic atleast 8-10 times within the nifi flow and is running correctly without any duplicated records.. 

Rashad_K_4-1759846196706.png

Can somebody please let me know what is the issue or am i missing something ?

Thank you!!

 

 

 

 

 

 

 

 

 

 

1 REPLY 1

avatar
Master Mentor

@Rashad_K 

What does NiFI provenance show for the duplicate inserts?
Do you see the same NiFi FlowFile (by FlowFile UUID) being successfully inserted?

Does the issue persist if your change the run duration from 25ms to 0ms?

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt